diff --git a/.gitattributes b/.gitattributes index a6344aac8c09253b3b630fb776ae94478aa0275b..4900d6bc7e906f9648cc3b3073c725701e43f758 100644 --- a/.gitattributes +++ b/.gitattributes @@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text *.zip filter=lfs diff=lfs merge=lfs -text *.zst filter=lfs diff=lfs merge=lfs -text *tfevents* filter=lfs diff=lfs merge=lfs -text +appendix.pdf filter=lfs diff=lfs merge=lfs -text +main.pdf filter=lfs diff=lfs merge=lfs -text +paper_results/figures/gains_compact.png filter=lfs diff=lfs merge=lfs -text +paper_results/figures/scaling_ranked.png filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000000000000000000000000000000000000..f04cca9401dc839053a8532ebf4d20ee5fcdceb1 --- /dev/null +++ b/README.md @@ -0,0 +1,47 @@ +--- +language: +- en +tags: +- computer-vision +- model-routing +- reproducibility +--- + +# ModelCollaboration: trained weights and paper results + +Code: https://github.com/nohi191212/ModelCollaboration + +## Contents + +- `weights.tar`: 16 main-table router checkpoints and configurations, training-set normalization statistics, the final 8M distilled image/text encoders, and the trained ConstructionSite YOLO26x and RT-DETR-X specialists. Detector extraction configurations record the checkpoint source and inference settings. RT-DETR uses FP32 inference in the recorded experiment. +- `experiment_data.tar.gz`: per-example validation/test learned scores and task outcomes for all 16 pairs, sample identifiers, selected validation thresholds, complete learned curves and the selection record. +- `paper_results/`: source data and scripts for paper tables and figures. `data/results.json` includes confidence and learned curves, fixed selected points and outcome decompositions. `data/main_table_cost_qwen_20260912/` contains the final Qwen cost curves and component measurements. Earlier ablations are explicitly separate from the main setting. +- `main.pdf` and `appendix.pdf`: the paper and supplementary material accompanying this release. + +## Recompute the reported routing results + +Download this repository, then extract the two archives into its root: + +```bash +tar -xf weights.tar +tar -xzf experiment_data.tar.gz +``` + +From the code repository, run: + +```bash +pip install numpy +python scripts/replay_paper_results.py --artifacts /path/to/download --output replayed_main.csv +``` + +This checks all saved learned validation/test curve points against the per-example outcomes, repeats validation-only operating-point selection, and exports the selected results. It does not execute the specialist or VLM. ConstructionSite uses event macro-F1; other tasks use their saved correctness indicators. + +To redraw the paper tables and outcome/budget figure, install matplotlib and run `paper_results/figures/build_tables.py` and `paper_results/figures/build_compact.py`. The source tables in `paper_results/tables/` retain the final typeset captions and labels. Cost data for the final figure are under `paper_results/data/main_table_cost_qwen_20260912/`; costs are assembled from measured components rather than a new timing run during replay. + +## Running the models again + +The weights archive contains `main/runs//best.pt`, its `config.json` and normalization arrays; `encoders/8M/epoch_300.pt`, tokenizer and encoder configuration; and `specialists//best.pt` with extraction settings. Match the 16 run names to `configs/main/` in the code repository. Preserve the frozen validation thresholds when evaluating test samples. + +Original dataset images, upstream VLM weights and full feature caches are external. Obtain upstream resources from the code repository's `docs/DOWNLOADS.md`. The full prepared feature cache is several GB and is excluded to keep non-weight experiment data below 100 MB. Running from original images requires preparing those features and predictions in the documented layout; the compact result replay above is the supported route for checking the published numbers without large-model inference. The release does not claim a one-command end-to-end reproduction from raw images. + +The original source datasets and upstream model terms continue to apply. This model card does not grant a new license for third-party datasets or detector architectures. diff --git a/appendix.pdf b/appendix.pdf new file mode 100644 index 0000000000000000000000000000000000000000..f09eff3a8b6f8cec2baf0b73ac2a7fd11b873120 --- /dev/null +++ b/appendix.pdf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f1f37970088137fb21832d56d5b793365463a54a291b05bca237aab50cf9a0ef +size 2430363 diff --git a/experiment_data.tar.gz b/experiment_data.tar.gz new file mode 100644 index 0000000000000000000000000000000000000000..5853f0785933c0c769a89963a801fac7840b75af --- /dev/null +++ b/experiment_data.tar.gz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7ee5a0063cae30a79812aff05d9302196687411c0441040252a375f3f6e2a5fa +size 2457041 diff --git a/main.pdf b/main.pdf new file mode 100644 index 0000000000000000000000000000000000000000..19b775b2be37fd8cb1e599d195cff4b393f8eb89 --- /dev/null +++ b/main.pdf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ac6d6f4cb7eb00f64e8e05789465e01aa865a1745dbcfb91dd0debb2344bf857 +size 577296 diff --git a/paper_results/data/budget_diagnostics.json b/paper_results/data/budget_diagnostics.json new file mode 100644 index 0000000000000000000000000000000000000000..48dba5aae5cb29302f3ac6d3aa16f48a654a7ba8 --- /dev/null +++ b/paper_results/data/budget_diagnostics.json @@ -0,0 +1,370 @@ +[ + { + "key": [ + "cub", + "cub", + "MiniCPM" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.9058333333333334, + "test_fixed_A25": 0.6472903003106663, + "test_ranked_A25_diagnostic": 0.7895926820849155, + "selected_validation_actual_calls": 0.01, + "selected_test_actual_calls": 0.113565757680359 + }, + "learned": { + "validation_fixed_A25": 0.9621666666666666, + "test_fixed_A25": 0.7763203313772868, + "test_ranked_A25_diagnostic": 0.7967552640662755, + "selected_validation_actual_calls": 0.01, + "selected_test_actual_calls": 0.028822920262340353 + } + } + }, + { + "key": [ + "cub", + "cub", + "Qwen3.8" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.9171666666666667, + "test_fixed_A25": 0.7316189161201243, + "test_ranked_A25_diagnostic": 0.8101484294097343, + "selected_validation_actual_calls": 0.01, + "selected_test_actual_calls": 0.113565757680359 + }, + "learned": { + "validation_fixed_A25": 0.9705833333333334, + "test_fixed_A25": 0.8053158439765276, + "test_ranked_A25_diagnostic": 0.806368657231619, + "selected_validation_actual_calls": 0.01, + "selected_test_actual_calls": 0.014325163962720055 + } + } + }, + { + "key": [ + "cub", + "glsim", + "MiniCPM" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.9005000000000001, + "test_fixed_A25": 0.7253365550569555, + "test_ranked_A25_diagnostic": 0.8519848118743528, + "selected_validation_actual_calls": 0.01, + "selected_test_actual_calls": 0.08698653779772178 + }, + "learned": { + "validation_fixed_A25": 0.97075, + "test_fixed_A25": 0.8740248532965137, + "test_ranked_A25_diagnostic": 0.8809112875388333, + "selected_validation_actual_calls": 0.03, + "selected_test_actual_calls": 0.04487400759406282 + } + } + }, + { + "key": [ + "cub", + "glsim", + "Qwen3.8" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.9213333333333334, + "test_fixed_A25": 0.7860545391784605, + "test_ranked_A25_diagnostic": 0.8691232309285468, + "selected_validation_actual_calls": 0.01, + "selected_test_actual_calls": 0.08698653779772178 + }, + "learned": { + "validation_fixed_A25": 0.9757500000000001, + "test_fixed_A25": 0.8918536416983087, + "test_ranked_A25_diagnostic": 0.8935623058336211, + "selected_validation_actual_calls": 0.01, + "selected_test_actual_calls": 0.016568864342423197 + } + } + }, + { + "key": [ + "grefcoco", + "groundingdino", + "MiniCPM" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.5566729917773561, + "test_fixed_A25": 0.27538212857669514, + "test_ranked_A25_diagnostic": 0.2786603522105323, + "selected_validation_actual_calls": 0.9899501019045611, + "selected_test_actual_calls": 0.9873805405098829 + }, + "learned": { + "validation_fixed_A25": 0.6255183076814955, + "test_fixed_A25": 0.33085386949493806, + "test_ranked_A25_diagnostic": 0.3352494115645294, + "selected_validation_actual_calls": 0.339939560053412, + "selected_test_actual_calls": 0.43518702322547714 + } + } + }, + { + "key": [ + "grefcoco", + "groundingdino", + "Qwen3.8" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.5648886077728582, + "test_fixed_A25": 0.28625187873975555, + "test_ranked_A25_diagnostic": 0.29324220854720245, + "selected_validation_actual_calls": 0.9899501019045611, + "selected_test_actual_calls": 0.9873805405098829 + }, + "learned": { + "validation_fixed_A25": 0.669126431934781, + "test_fixed_A25": 0.3549726342058248, + "test_ranked_A25_diagnostic": 0.37351898590590704, + "selected_validation_actual_calls": 0.4199873497786211, + "selected_test_actual_calls": 0.5745682443354224 + } + } + }, + { + "key": [ + "grefcoco", + "instancevg", + "MiniCPM" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.7465176751704266, + "test_fixed_A25": 0.6639253608598248, + "test_ranked_A25_diagnostic": 0.6590023537418825, + "selected_validation_actual_calls": 0.6499402628434886, + "selected_test_actual_calls": 0.5209426310864078 + }, + "learned": { + "validation_fixed_A25": 0.7607140347178298, + "test_fixed_A25": 0.6759152652922327, + "test_ranked_A25_diagnostic": 0.6777273629583416, + "selected_validation_actual_calls": 0.22995291306486754, + "selected_test_actual_calls": 0.30904914499617164 + } + } + }, + { + "key": [ + "grefcoco", + "instancevg", + "Qwen3.8" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.7477616136060159, + "test_fixed_A25": 0.6640728242066756, + "test_ranked_A25_diagnostic": 0.6611405722712191, + "selected_validation_actual_calls": 0.7099585353854804, + "selected_test_actual_calls": 0.6838612710206166 + }, + "learned": { + "validation_fixed_A25": 0.7778129172816081, + "test_fixed_A25": 0.7280974392422653, + "test_ranked_A25_diagnostic": 0.7136828970875989, + "selected_validation_actual_calls": 0.5599831330381615, + "selected_test_actual_calls": 0.5355755324277571 + } + } + }, + { + "key": [ + "nlvr2", + "nlvr2", + "MiniCPM" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.8284302492122602, + "test_fixed_A25": 0.833902561526871, + "test_ranked_A25_diagnostic": 0.835882901628758, + "selected_validation_actual_calls": 0.8587797192781438, + "selected_test_actual_calls": 0.8580038745784602 + }, + "learned": { + "validation_fixed_A25": 0.8359782297336007, + "test_fixed_A25": 0.8425629618999785, + "test_ranked_A25_diagnostic": 0.8414221137978044, + "selected_validation_actual_calls": 0.8498997421942137, + "selected_test_actual_calls": 0.8465953935567195 + } + } + }, + { + "key": [ + "nlvr2", + "nlvr2", + "Qwen3.8" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.831280435405328, + "test_fixed_A25": 0.8343976465523427, + "test_ranked_A25_diagnostic": 0.8361340317141422, + "selected_validation_actual_calls": 0.8587797192781438, + "selected_test_actual_calls": 0.8580038745784602 + }, + "learned": { + "validation_fixed_A25": 0.8402320252076769, + "test_fixed_A25": 0.8436320585491857, + "test_ranked_A25_diagnostic": 0.8425342613187918, + "selected_validation_actual_calls": 0.7699799484388428, + "selected_test_actual_calls": 0.7665925234986009 + } + } + }, + { + "key": [ + "nlvr2", + "vilt", + "MiniCPM" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.7926095674591808, + "test_fixed_A25": 0.7987371744277822, + "test_ranked_A25_diagnostic": 0.8005237856066586, + "selected_validation_actual_calls": 0.9598968776854769, + "selected_test_actual_calls": 0.9593169261677549 + }, + "learned": { + "validation_fixed_A25": 0.793425952449155, + "test_fixed_A25": 0.8005094353160652, + "test_ranked_A25_diagnostic": 0.8015211308028988, + "selected_validation_actual_calls": 0.9899742194213692, + "selected_test_actual_calls": 0.9896677907727631 + } + } + }, + { + "key": [ + "nlvr2", + "vilt", + "Qwen3.8" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.7940561443712404, + "test_fixed_A25": 0.7988735021884195, + "test_ranked_A25_diagnostic": 0.8006242376408124, + "selected_validation_actual_calls": 0.9598968776854769, + "selected_test_actual_calls": 0.9593169261677549 + }, + "learned": { + "validation_fixed_A25": 0.7963477513606417, + "test_fixed_A25": 0.802590227452106, + "test_ranked_A25_diagnostic": 0.8037526009901701, + "selected_validation_actual_calls": 0.9498710971068461, + "selected_test_actual_calls": 0.9485542082227165 + } + } + }, + { + "key": [ + "construction", + "yolo26x", + "MiniCPM" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.17451640525159154, + "test_fixed_A25": 0.17697501725814352, + "test_ranked_A25_diagnostic": 0.1716572124279524, + "selected_validation_actual_calls": 0.9386590584878745, + "selected_test_actual_calls": 0.9254327563249002 + }, + "learned": { + "validation_fixed_A25": 0.22914536623152232, + "test_fixed_A25": 0.21733392403576846, + "test_ranked_A25_diagnostic": 0.21579610594149268, + "selected_validation_actual_calls": 0.4893009985734665, + "selected_test_actual_calls": 0.47203728362183756 + } + } + }, + { + "key": [ + "construction", + "yolo26x", + "Qwen3.8" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.17942553980644182, + "test_fixed_A25": 0.18055274894913714, + "test_ranked_A25_diagnostic": 0.1793307821359281, + "selected_validation_actual_calls": 0.9286733238231099, + "selected_test_actual_calls": 0.9221038615179761 + }, + "learned": { + "validation_fixed_A25": 0.23466194499464404, + "test_fixed_A25": 0.21316340540985348, + "test_ranked_A25_diagnostic": 0.213003148312119, + "selected_validation_actual_calls": 0.4992867332382311, + "selected_test_actual_calls": 0.4876830892143808 + } + } + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "MiniCPM" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.08994758708890332, + "test_fixed_A25": 0.08325267500178911, + "test_ranked_A25_diagnostic": 0.08357421552875581, + "selected_validation_actual_calls": 0.978601997146933, + "selected_test_actual_calls": 0.9790279627163782 + }, + "learned": { + "validation_fixed_A25": 0.11165309347543784, + "test_fixed_A25": 0.09726595269708746, + "test_ranked_A25_diagnostic": 0.09795552217467457, + "selected_validation_actual_calls": 0.7788873038516405, + "selected_test_actual_calls": 0.7882822902796272 + } + } + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "Qwen3.8" + ], + "areas": { + "confidence": { + "validation_fixed_A25": 0.10053453664183931, + "test_fixed_A25": 0.08910633276678806, + "test_ranked_A25_diagnostic": 0.08907503728424503, + "selected_validation_actual_calls": 0.9885877318116976, + "selected_test_actual_calls": 0.9860186418109188 + }, + "learned": { + "validation_fixed_A25": 0.13205087205171664, + "test_fixed_A25": 0.10226629753438624, + "test_ranked_A25_diagnostic": 0.10459671104829708, + "selected_validation_actual_calls": 0.8288159771754636, + "selected_test_actual_calls": 0.831225033288948 + } + } + } +] \ No newline at end of file diff --git a/paper_results/data/comparisons.json b/paper_results/data/comparisons.json new file mode 100644 index 0000000000000000000000000000000000000000..252c5a252c8914910bb85432e19920c97ef18024 --- /dev/null +++ b/paper_results/data/comparisons.json @@ -0,0 +1,11558 @@ +[ + { + "baseline": "original", + "candidate": "score_gain", + "counts": { + "total": 48, + "performance_wins": 31, + "performance_ties": 0, + "performance_losses": 17, + "call_less": 21, + "call_ties": 0, + "call_more": 27, + "both_performance_win_and_call_less": 19, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.23610378506203042, + "performance_delta": 0.00636626667272841, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7952729693741678, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23330554571941767, + "performance_delta": 0.0008511922008775652, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7902796271637816, + "call_delta": 0.009320905459387463 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.22951409007632145, + "performance_delta": -0.003254425173411568, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.8009320905459387, + "call_delta": 0.019640479360852225 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2687941772737477, + "performance_delta": 0.0004707717644751308, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8062583222370173, + "call_delta": -0.00998668442077233 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2699197889196931, + "performance_delta": 0.003992751149164753, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.831225033288948, + "call_delta": -0.007989347536617841 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.2687146609089476, + "performance_delta": 0.00037410281989919136, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8225699067909454, + "call_delta": -0.006990679094540542 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2668710925054341, + "performance_delta": -0.0004626043850421979, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.39580559254327563, + "call_delta": 0.023968042609853524 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.25610142784250745, + "performance_delta": -0.002510607023462952, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.46804260985352863, + "call_delta": 0.10852197070572572 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2553143923177106, + "performance_delta": -0.010255108607445773, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5402796271637816, + "call_delta": 0.08588548601864182 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.3034889605344151, + "performance_delta": 0.005719274933867702, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.46937416777629826, + "call_delta": -0.021637816245006658 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2998980207837558, + "performance_delta": 0.000508977878148853, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.4284287616511318, + "call_delta": -0.025965379494008012 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3033337683996216, + "performance_delta": 0.0003620662089585114, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.45272969374167776, + "call_delta": -0.001997336884154488 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0024162927166033743, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.004832585433206766, + "call_delta": -0.0372799447704522 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.8134276838108389, + "performance_delta": 0.005522954780807776, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.010700724887814981, + "call_delta": -0.04470141525716258 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.81429064549534, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.0075940628236106315, + "call_delta": -0.03676216775975147 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0008629616845011734, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.0075940628236106315, + "call_delta": -0.010182947877114256 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0012081463583016872, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.0069036934760096655, + "call_delta": -0.006213324128408699 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8151536071798412, + "performance_delta": 0.0008629616845011734, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.007766655160510874, + "call_delta": -0.0006903693476009659 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.8998964445978599, + "performance_delta": 0.0006903693476010275, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.027614773904038662, + "call_delta": 0.001208146358301694 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.8995512599240594, + "performance_delta": 0.0010355540214014303, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.016396272005522954, + "call_delta": -0.007594062823610633 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8988608905764585, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.015533310321021747, + "call_delta": -0.00655850880220918 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0020711080428028605, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.00845702450811184, + "call_delta": -0.010873317224715224 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0006903693476009165, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.009492578529513289, + "call_delta": -0.00742147048671039 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.9011045909561616, + "performance_delta": 0.0008629616845012844, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.011045909561615464, + "call_delta": -0.004487400759406283 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48586336953747555, + "performance_delta": 0.029265802682698572, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5045798712531548, + "call_delta": 0.09743924226526385 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.49794402064486853, + "performance_delta": 0.034767319853670986, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.5430054164421632, + "call_delta": 0.08740039134503585 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.49649774551229336, + "performance_delta": 0.05016589626520718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5379576326461163, + "call_delta": 0.13382298726710712 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7514108272126592, + "performance_delta": 0.002070158523097887, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.8495023112043786, + "call_delta": -0.00944332586563812 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.752006352267249, + "performance_delta": 0.039389728610725205, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8824263392224144, + "call_delta": 0.19391430110881092 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7502197771034795, + "performance_delta": 0.09576610044522582, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9653177551541275, + "call_delta": 0.4029435952698296 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6800045373337492, + "performance_delta": 0.005388083827240986, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.12287666959702805, + "call_delta": -0.10997362674758245 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6805717040524062, + "performance_delta": 0.0093582508578397, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.09480191702350906, + "call_delta": -0.18211723336074642 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6810254374273318, + "performance_delta": 0.007032867311346225, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.12109009443325866, + "call_delta": -0.15049768879562145 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7459376683776197, + "performance_delta": -0.012619459490117113, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.978617814706633, + "call_delta": 0.4110824376825568 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7473555851742619, + "performance_delta": -0.011003034341944828, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.968380455434875, + "call_delta": 0.35853444119899036 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7482063352522474, + "performance_delta": -0.012194084451124354, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.96633865524771, + "call_delta": 0.36423446672149273 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9067231111430006, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9916050800028701, + "call_delta": 0.06845088613044414 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9069383655019014, + "performance_delta": -0.0010762717945038691, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9886632704312263, + "call_delta": 0.05431584989596039 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9057185908014638, + "performance_delta": -0.0018655377771399362, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9883045131663917, + "call_delta": 0.12004018081366152 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9060055966133315, + "performance_delta": -0.001004520341536863, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9787615699217909, + "call_delta": 0.12850685226375824 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9049293248188276, + "performance_delta": 0.0011480232474707641, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9672813374470833, + "call_delta": 0.10303508646050086 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9067948625959675, + "performance_delta": -0.00035875726483469705, + "baseline_call": 0.98880677333716, + "candidate_call": 0.9913180741910024, + "call_delta": 0.0025113008538423243 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.906364353878166, + "performance_delta": -0.0006457630767022771, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9926813517973739, + "call_delta": 0.002009040683073837 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.989811293678697, + "call_delta": 0.0025830523068092193 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9039965559302576, + "performance_delta": 0.004018081366147674, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9788333213747579, + "call_delta": 0.017148597259094567 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9023462725120184, + "performance_delta": 0.0022960464949415282, + "baseline_call": 0.969433881036091, + "candidate_call": 0.9800530960751955, + "call_delta": 0.010619215039104568 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.9036377986654229, + "performance_delta": 0.0033005668364783913, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9882327617134247, + "call_delta": 0.017722608882829838 + } + ], + "rule": "historical" + }, + { + "baseline": "original", + "candidate": "checkpoint_perf", + "counts": { + "total": 48, + "performance_wins": 12, + "performance_ties": 10, + "performance_losses": 26, + "call_less": 20, + "call_ties": 8, + "call_more": 20, + "both_performance_win_and_call_less": 7, + "both_performance_tie_and_call_less": 1 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.2305421865004371, + "performance_delta": 0.0008046681111350817, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7816245006657789, + "call_delta": -0.00898801597869514 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.22838091389772372, + "performance_delta": -0.0040734396208163826, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.8049267643142477, + "call_delta": 0.023968042609853524 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.23035654564059277, + "performance_delta": -0.002411969609140252, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.774300932090546, + "call_delta": -0.006990679094540542 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.26792218076116586, + "performance_delta": -0.00040122474810672104, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8215712383488681, + "call_delta": 0.005326231691078487 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.26876666264349774, + "performance_delta": 0.002839624872969404, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8125832223701731, + "call_delta": -0.026631158455392767 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.26833406705021806, + "performance_delta": -6.491038830325202e-06, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8229027962716379, + "call_delta": -0.0066577896138481085 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2624005680779394, + "performance_delta": -0.0049331288125369355, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.37350199733688416, + "call_delta": 0.0016644474034620549 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.25991507502469235, + "performance_delta": 0.0013030401587219576, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.37283621837549935, + "call_delta": 0.013315579227696439 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2613858995371947, + "performance_delta": -0.004183601387961644, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.3808255659121172, + "call_delta": -0.07356857523302263 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.29875083466112995, + "performance_delta": 0.0009811490605825535, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.43275632490013316, + "call_delta": -0.058255659121171754 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.29645395142312325, + "performance_delta": -0.0029350914824837204, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.524300932090546, + "call_delta": 0.06990679094540614 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3014472023134628, + "performance_delta": -0.0015244998772003249, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.4490679094540613, + "call_delta": -0.0056591211717709755 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.8129099068001381, + "performance_delta": 0.0010355540214014303, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.05074214704867104, + "call_delta": 0.00862961684501208 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.8130824991370383, + "performance_delta": 0.005177770107007262, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.044183638246461855, + "call_delta": -0.011218501898515705 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.8151536071798412, + "performance_delta": 0.0, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.0443562305833621, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.8123921297894373, + "performance_delta": -0.0010355540214015413, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.020711080428028994, + "call_delta": 0.002934069727304107 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.8127373144632378, + "performance_delta": -0.00034518467380051376, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.02347255781843286, + "call_delta": 0.010355540214014497 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8113565757680359, + "performance_delta": -0.002934069727304145, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.03641698308595098, + "call_delta": 0.027959958577839145 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.8967897825336555, + "performance_delta": -0.0024162927166033743, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.038488091128753885, + "call_delta": 0.012081463583016917 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.8979979288919572, + "performance_delta": -0.0005177770107007706, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.0277873662409389, + "call_delta": 0.003797031411805315 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8966171901967552, + "performance_delta": -0.003106662064204402, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.03141180531584398, + "call_delta": 0.00931998619261305 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.9009319986192613, + "performance_delta": 0.001380738695201944, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.018985157059026578, + "call_delta": -0.000345184673800486 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.9005868139454608, + "performance_delta": -0.00034518467380051376, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.015015533310321022, + "call_delta": -0.0018985157059026574 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.8992060752502589, + "performance_delta": -0.0010355540214014303, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.015360717984121504, + "call_delta": -0.000172592336900243 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.445622890848765, + "performance_delta": -0.010974676006011985, + "baseline_call": 0.407140628987891, + "candidate_call": 0.4267929557893543, + "call_delta": 0.019652326801463282 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.4702379264384766, + "performance_delta": 0.007061225647279068, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.47905736891359213, + "call_delta": 0.023452343816464827 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.44219153220089047, + "performance_delta": -0.004140317046195718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.4057227121912486, + "call_delta": 0.0015880668122394415 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.6876612880356181, + "performance_delta": -0.06167938065394318, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.6266057907721975, + "call_delta": -0.23233984629781923 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.6622805773757195, + "performance_delta": -0.050336046280804236, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.5790488614128123, + "call_delta": -0.1094631767007912 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.6775940787794572, + "performance_delta": 0.023140402121203496, + "baseline_call": 0.562374159884298, + "candidate_call": 0.6149505147037971, + "call_delta": 0.05257635481949918 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6657119360235941, + "performance_delta": -0.008904517482914098, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.322916371267334, + "call_delta": 0.09006607492272353 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6712134531945665, + "performance_delta": 0.0, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.27691915038425546, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6739925701159856, + "performance_delta": 0.0, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.2715877832288801, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7582735445084082, + "performance_delta": -0.00028358335932854306, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.537475540935258, + "call_delta": -0.030059836088818237 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7583586195162068, + "performance_delta": 0.0, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.6098460142358847, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7604004197033718, + "performance_delta": 0.0, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6021041885262173, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9080863887493722, + "performance_delta": 0.0, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9231541938724259, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9079428858434383, + "performance_delta": -7.175145296700602e-05, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9026332783238861, + "call_delta": -0.0317141422113798 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.907297122766736, + "performance_delta": -0.00028700581186769103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8648202626103179, + "call_delta": -0.0034440697424122924 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9072253713137691, + "performance_delta": -0.0009327688885699681, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.8425773121905719, + "call_delta": 0.01528305948195452 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9070101169548683, + "performance_delta": 0.0, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.8502547176580326, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9037813015713568, + "performance_delta": 0.0, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8642462509865825, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9071536198608022, + "performance_delta": 0.0, + "baseline_call": 0.98880677333716, + "candidate_call": 0.9778288010332209, + "call_delta": -0.010977972303939154 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9070818684078352, + "performance_delta": 7.1751452966895e-05, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9764655234268494, + "call_delta": -0.01420678768745065 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9047858219128937, + "performance_delta": -0.0023677979479085343, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9235129511372605, + "call_delta": -0.0637152902346273 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9013417521704814, + "performance_delta": 0.0013632776063714491, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9797660902633278, + "call_delta": 0.018081366147664535 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9002654803759776, + "performance_delta": 0.00021525435890079603, + "baseline_call": 0.969433881036091, + "candidate_call": 0.966563822917414, + "call_delta": -0.0028700581186769103 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.9003372318289445, + "performance_delta": 0.0, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9903853053024324, + "call_delta": 0.019875152471837576 + } + ], + "rule": "historical" + }, + { + "baseline": "original", + "candidate": "both", + "counts": { + "total": 48, + "performance_wins": 26, + "performance_ties": 2, + "performance_losses": 20, + "call_less": 24, + "call_ties": 1, + "call_more": 23, + "both_performance_win_and_call_less": 16, + "both_performance_tie_and_call_less": 1 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22850906574549645, + "performance_delta": -0.0012284526438055554, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7906125166444741, + "call_delta": 0.0 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23004315338506715, + "performance_delta": -0.002411200133472957, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7926098535286284, + "call_delta": 0.011651131824234273 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.2308450652014655, + "performance_delta": -0.001923450048267522, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.7766311584553928, + "call_delta": -0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2678997047385957, + "performance_delta": -0.0004237007706768825, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8209054593874834, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2672954581839896, + "performance_delta": 0.0013684204134612465, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8215712383488681, + "call_delta": -0.017643142476697737 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.267918819975632, + "performance_delta": -0.0004217381134163922, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8149134487350199, + "call_delta": -0.01464713715046606 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2675187098549581, + "performance_delta": 0.00018501296448181215, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.4144474034620506, + "call_delta": 0.042609853528628505 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2578686516978949, + "performance_delta": -0.0007433831680754843, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.35918774966711053, + "call_delta": -0.00033288948069237767 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.26086905351963807, + "performance_delta": -0.004700447405518282, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.39980026631158455, + "call_delta": -0.05459387483355527 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.3029107097560361, + "performance_delta": 0.005141024155488727, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.4337549933422104, + "call_delta": -0.05725699067909451 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.29937410202312603, + "performance_delta": -1.4940882480940498e-05, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.44274300932090543, + "call_delta": -0.011651131824234384 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3009613147999525, + "performance_delta": -0.002010387390710633, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.4477363515312916, + "call_delta": -0.006990679094540653 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.8125647221263376, + "performance_delta": 0.0006903693476009165, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.017604418363824648, + "call_delta": -0.024508111839834312 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.8134276838108389, + "performance_delta": 0.005522954780807776, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.010700724887814981, + "call_delta": -0.04470141525716258 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.81429064549534, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.006558508802209182, + "call_delta": -0.03779772178115291 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0008629616845011734, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.007766655160510874, + "call_delta": -0.010010355540214014 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0012081463583016872, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.005695547117707974, + "call_delta": -0.00742147048671039 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.00638591646530894, + "call_delta": -0.0020711080428028996 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.8998964445978599, + "performance_delta": 0.0006903693476010275, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.027614773904038662, + "call_delta": 0.001208146358301694 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.901449775629962, + "performance_delta": 0.002934069727304034, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.010873317224715223, + "call_delta": -0.013117017604418364 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.901449775629962, + "performance_delta": 0.0017259233690023468, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.010355540214014497, + "call_delta": -0.01173627890921643 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0020711080428028605, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.01156368657231619, + "call_delta": -0.0077666551605108745 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0006903693476009165, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.006731101139109423, + "call_delta": -0.010182947877114256 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.001380738695201944, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.007076285812909907, + "call_delta": -0.008457024508111841 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48586336953747555, + "performance_delta": 0.029265802682698572, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5045798712531548, + "call_delta": 0.09743924226526385 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.49794402064486853, + "performance_delta": 0.034767319853670986, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.5430054164421632, + "call_delta": 0.08740039134503585 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.49649774551229336, + "performance_delta": 0.05016589626520718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5379576326461163, + "call_delta": 0.13382298726710712 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7511272438533307, + "performance_delta": 0.001786575163769455, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.852763519836656, + "call_delta": -0.006182117233360707 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7536511357513541, + "performance_delta": 0.04103451209483033, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8933726568924936, + "call_delta": 0.20486061877889006 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7511839605251964, + "performance_delta": 0.09673028386694271, + "baseline_call": 0.562374159884298, + "candidate_call": 0.8652411876471089, + "call_delta": 0.30286702776281094 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6800045373337492, + "performance_delta": 0.005388083827240986, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.12287666959702805, + "call_delta": -0.10997362674758245 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6805717040524062, + "performance_delta": 0.0093582508578397, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.09480191702350906, + "call_delta": -0.18211723336074642 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6810254374273318, + "performance_delta": 0.007032867311346225, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.12109009443325866, + "call_delta": -0.15049768879562145 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7519496355953833, + "performance_delta": -0.006607492272353466, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.8643620792331906, + "call_delta": 0.29682670220911445 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7473555851742619, + "performance_delta": -0.011003034341944828, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9705640473017043, + "call_delta": 0.3607180330658196 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7550406942120637, + "performance_delta": -0.005359725491308143, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9094518333664181, + "call_delta": 0.30734764484020083 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9060773480662984, + "performance_delta": -0.002009040683073837, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9890937791490277, + "call_delta": 0.06593958527660182 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0015785319652723562, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9886632704312263, + "call_delta": 0.05431584989596039 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9071536198608022, + "performance_delta": -0.00043050871780148103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.98973954222573, + "call_delta": 0.12147520987299987 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9052163306306953, + "performance_delta": -0.0017937863241730412, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9674248403530171, + "call_delta": 0.1171701226949845 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9075123771256368, + "performance_delta": 0.0037310755542799834, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9586711630910526, + "call_delta": 0.09442491210447013 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9071536198608022, + "performance_delta": 0.0, + "baseline_call": 0.98880677333716, + "candidate_call": 0.99253784889144, + "call_delta": 0.0037310755542799834 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9066513596900337, + "performance_delta": -0.00035875726483458603, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9929683576092416, + "call_delta": 0.0022960464949415282 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.989811293678697, + "call_delta": 0.0025830523068092193 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9034225443065222, + "performance_delta": 0.0034440697424122924, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9885197675252924, + "call_delta": 0.026835043409629056 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9037095501183899, + "performance_delta": 0.0036593241013130884, + "baseline_call": 0.969433881036091, + "candidate_call": 0.9783310612039894, + "call_delta": 0.008897180167898422 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.9026332783238861, + "performance_delta": 0.0022960464949415282, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9682141063356533, + "call_delta": -0.0022960464949415282 + } + ], + "rule": "historical" + }, + { + "baseline": "original", + "candidate": "score_only_gain", + "counts": { + "total": 48, + "performance_wins": 20, + "performance_ties": 1, + "performance_losses": 27, + "call_less": 17, + "call_ties": 0, + "call_more": 31, + "both_performance_win_and_call_less": 10, + "both_performance_tie_and_call_less": 1 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22725460966786967, + "performance_delta": -0.0024829087214323364, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.8768308921438083, + "call_delta": 0.0862183754993342 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23264549293538322, + "performance_delta": 0.0001911394168431224, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7882822902796272, + "call_delta": 0.007323568575232975 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.229420185942548, + "performance_delta": -0.0033483293071850107, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.7959387483355526, + "call_delta": 0.01464713715046606 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2678997047385957, + "performance_delta": -0.0004237007706768825, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8209054593874834, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.270279514537055, + "performance_delta": 0.0043524767665266695, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.801930758988016, + "call_delta": -0.03728362183754985 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.26989568494530464, + "performance_delta": 0.0015551268562562504, + "baseline_call": 0.829560585885486, + "candidate_call": 0.800266311584554, + "call_delta": -0.02929427430093201 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2668710925054341, + "performance_delta": -0.0004626043850421979, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.39580559254327563, + "call_delta": 0.023968042609853524 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2578686516978949, + "performance_delta": -0.0007433831680754843, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.35918774966711053, + "call_delta": -0.00033288948069237767 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2565642588848189, + "performance_delta": -0.00900524204033748, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.46571238348868177, + "call_delta": 0.011318242343541951 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.29895985090664157, + "performance_delta": 0.0011901653060941708, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.4497336884154461, + "call_delta": -0.04127829560585883 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2991984700035601, + "performance_delta": -0.0001905729020468816, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.453728362183755, + "call_delta": -0.0006657789613848109 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3033337683996216, + "performance_delta": 0.0003620662089585114, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.45272969374167776, + "call_delta": -0.001997336884154488 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.8125647221263376, + "performance_delta": 0.0006903693476009165, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.026406627545736968, + "call_delta": -0.015705902657921992 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.8111839834311356, + "performance_delta": 0.0032792544011045477, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.02347255781843286, + "call_delta": -0.0319295823265447 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.81429064549534, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.006558508802209182, + "call_delta": -0.03779772178115291 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.8134276838108389, + "performance_delta": 0.0, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.01570590265792199, + "call_delta": -0.0020711080428028987 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.8118743527787367, + "performance_delta": -0.0012081463583016872, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.0298584742837418, + "call_delta": 0.016741456679323436 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8127373144632378, + "performance_delta": -0.001553331032102201, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.032274767000345185, + "call_delta": 0.023817742492233347 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.9005868139454608, + "performance_delta": 0.001380738695201944, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.021574042112530202, + "call_delta": -0.004832585433206766 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.8973075595443563, + "performance_delta": -0.0012081463583016872, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.02347255781843286, + "call_delta": -0.0005177770107007255 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8992060752502589, + "performance_delta": -0.0005177770107007706, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.020711080428028994, + "call_delta": -0.0013807386952019336 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.8992060752502589, + "performance_delta": -0.00034518467380051376, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.022091819123230928, + "call_delta": 0.0027614773904038638 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.8990334829133586, + "performance_delta": -0.0018985157059027147, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.02002071108042803, + "call_delta": 0.00310666206420435 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.8988608905764585, + "performance_delta": -0.001380738695201833, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.02105626510182948, + "call_delta": 0.005522954780807733 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.511414230212971, + "performance_delta": 0.05481666335819407, + "baseline_call": 0.407140628987891, + "candidate_call": 0.7920199642684967, + "call_delta": 0.38487933528060575 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.503984346198565, + "performance_delta": 0.04080764540736748, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.597339988089499, + "call_delta": 0.14173496299237165 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5101097467600602, + "performance_delta": 0.06377789751297397, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.7076255565323427, + "call_delta": 0.30349091115333354 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7498794770722854, + "performance_delta": 0.0005388083827241319, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.846070952556504, + "call_delta": -0.012874684513512702 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7536511357513541, + "performance_delta": 0.04103451209483033, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8933726568924936, + "call_delta": 0.20486061877889006 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7510138105095993, + "performance_delta": 0.09656013385134554, + "baseline_call": 0.562374159884298, + "candidate_call": 0.8730964467005076, + "call_delta": 0.3107222868162096 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6809403624195333, + "performance_delta": 0.006323908913025034, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.1506961971471514, + "call_delta": -0.08215409919745909 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6708164364915067, + "performance_delta": -0.00039701670305980485, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.00870600913138417, + "call_delta": -0.26821314125287127 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6797493123103536, + "performance_delta": 0.0057567421943680586, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.11924680259762357, + "call_delta": -0.15234098063125653 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7519496355953833, + "performance_delta": -0.006607492272353466, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.8643620792331906, + "call_delta": 0.29682670220911445 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7474406601820606, + "performance_delta": -0.010917959334146188, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9800073731673425, + "call_delta": 0.37016135893145785 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7482063352522474, + "performance_delta": -0.012194084451124354, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.96633865524771, + "call_delta": 0.36423446672149273 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9068666140489345, + "performance_delta": -0.0012197747004376591, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9918203343617709, + "call_delta": 0.06866614048934494 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9055750878955299, + "performance_delta": -0.0024395494008754293, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9716581760780656, + "call_delta": 0.03731075554279972 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9057185908014638, + "performance_delta": -0.0018655377771399362, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9883045131663917, + "call_delta": 0.12004018081366152 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9060055966133315, + "performance_delta": -0.001004520341536863, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9787615699217909, + "call_delta": 0.12850685226375824 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9060773480662984, + "performance_delta": 0.0022960464949415282, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9367152184831743, + "call_delta": 0.07246896749659182 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9060773480662984, + "performance_delta": -0.0010762717945038691, + "baseline_call": 0.98880677333716, + "candidate_call": 0.9899547965846308, + "call_delta": 0.0011480232474707641 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.906364353878166, + "performance_delta": -0.0006457630767022771, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9926813517973739, + "call_delta": 0.002009040683073837 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9059338451603645, + "performance_delta": -0.0012197747004377701, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9891655306019946, + "call_delta": 0.0019372892301068312 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9033507928535552, + "performance_delta": 0.0033723182894452863, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9880892588074909, + "call_delta": 0.026404534691827575 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9023462725120184, + "performance_delta": 0.0022960464949415282, + "baseline_call": 0.969433881036091, + "candidate_call": 0.9800530960751955, + "call_delta": 0.010619215039104568 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.9033507928535552, + "performance_delta": 0.0030135610246107003, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.987012987012987, + "call_delta": 0.01650283418239218 + } + ], + "rule": "historical" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_0.25", + "counts": { + "total": 48, + "performance_wins": 17, + "performance_ties": 0, + "performance_losses": 31, + "call_less": 5, + "call_ties": 0, + "call_more": 43, + "both_performance_win_and_call_less": 1, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.2211131170053352, + "performance_delta": -0.008624401383966807, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.988681757656458, + "call_delta": 0.19806924101198398 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.2222578228236749, + "performance_delta": -0.010196530694865202, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.9880159786950732, + "call_delta": 0.207057256990679 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.22347202518399215, + "performance_delta": -0.009296490065740876, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.988681757656458, + "call_delta": 0.20739014647137155 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2707502617838614, + "performance_delta": 0.002426856274588829, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.9880159786950732, + "call_delta": 0.17177097203728353 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.27188089881706984, + "performance_delta": 0.005953861046541498, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.9883488681757656, + "call_delta": 0.14913448735019974 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.27098320199585757, + "performance_delta": 0.0026426439068091834, + "baseline_call": 0.829560585885486, + "candidate_call": 0.9873501997336884, + "call_delta": 0.15778961384820245 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2587284729338324, + "performance_delta": -0.008605223956643926, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.5702396804260985, + "call_delta": 0.1984021304926764 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2591386172712909, + "performance_delta": 0.000526582405320497, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.5509320905459387, + "call_delta": 0.1914114513981358 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.25650251356280634, + "performance_delta": -0.00906698736235001, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5622503328894807, + "call_delta": 0.10785619174434086 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.2712680480298825, + "performance_delta": -0.02650163757066487, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.6844207723035952, + "call_delta": 0.1934087882822903 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.27348865863056027, + "performance_delta": -0.025900384275046706, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.6474700399467377, + "call_delta": 0.19307589880159787 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2704420670349152, + "performance_delta": -0.032529635155747916, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.5572569906790945, + "call_delta": 0.10252996005326226 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.804280289955126, + "performance_delta": -0.007594062823610637, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.10631687953054884, + "call_delta": 0.06420434932688988 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.8034173282706247, + "performance_delta": -0.004487400759406346, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.1102865032792544, + "call_delta": 0.054884363134276845 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.8030721435968243, + "performance_delta": -0.012081463583016872, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.11408353469105972, + "call_delta": 0.06972730410769762 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.8203313772868485, + "performance_delta": 0.006903693476009609, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.10976872626855368, + "call_delta": 0.09199171556782879 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.8198136002761477, + "performance_delta": 0.006731101139109352, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.10631687953054884, + "call_delta": 0.09319986192613047 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8194684156023473, + "performance_delta": 0.005177770107007262, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.10976872626855368, + "call_delta": 0.10131170176044184 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.8779772178115292, + "performance_delta": -0.02122885743872971, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.09009319986192613, + "call_delta": 0.06368657231618916 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.8765964791163272, + "performance_delta": -0.021919226786330737, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.09199171556782879, + "call_delta": 0.0680013807386952 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8781498101484294, + "performance_delta": -0.021574042112530223, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.08940283051432517, + "call_delta": 0.06731101139109424 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.8893683120469451, + "performance_delta": -0.010182947877114268, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.08940283051432517, + "call_delta": 0.0700724887814981 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.8881601656886434, + "performance_delta": -0.012771832930617899, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.09354504659993096, + "call_delta": 0.07663099758370728 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.8885053503624439, + "performance_delta": -0.011736278909216358, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.09112875388332758, + "call_delta": 0.07559544356230584 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.4992201457618467, + "performance_delta": 0.04262257890706972, + "baseline_call": 0.407140628987891, + "candidate_call": 0.9824178317216345, + "call_delta": 0.5752772027337436 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.4999858208320336, + "performance_delta": 0.03680912004083603, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.9854238153305164, + "call_delta": 0.5298187902333891 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.4995604457930409, + "performance_delta": 0.05322859654595469, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.984941723619658, + "call_delta": 0.5808070782406489 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7393018177693333, + "performance_delta": -0.010038850920227937, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.9836088818308142, + "call_delta": 0.12466324476079749 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7390182344100048, + "performance_delta": 0.02640161075348102, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.9834954484870828, + "call_delta": 0.2949834103734793 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7399540594957887, + "performance_delta": 0.08550038283753503, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9844879902447324, + "call_delta": 0.4221138303604345 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6695403113745285, + "performance_delta": -0.005076142131979711, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.019595610129597596, + "call_delta": -0.2132546862150129 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6694835947026628, + "performance_delta": -0.0017298584919036575, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.05478830502226129, + "call_delta": -0.22213084536199418 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6695403113745285, + "performance_delta": -0.004452258741457049, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.017383659926835493, + "call_delta": -0.2542041233020446 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7470436434790007, + "performance_delta": -0.011513484388736117, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.9068712247965289, + "call_delta": 0.33933584777245274 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7490287269943, + "performance_delta": -0.009329892521906746, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9408445112440802, + "call_delta": 0.3309984970081955 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7492272353458299, + "performance_delta": -0.011173184357541888, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.938831069392848, + "call_delta": 0.3367268808666307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9071536198608022, + "performance_delta": -0.0009327688885699681, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9870847384659539, + "call_delta": 0.06393054459352798 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9050728277247614, + "performance_delta": -0.0029418095716439163, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9550835904427064, + "call_delta": 0.020736169907440538 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9062926024251992, + "performance_delta": -0.0012915261534045541, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9650570424051087, + "call_delta": 0.09679271005237855 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9090191576379422, + "performance_delta": 0.0008610174356030731, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.9368587213891081, + "call_delta": 0.10956446868049075 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9069383655019014, + "performance_delta": -7.1751452966895e-05, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9668508287292817, + "call_delta": 0.11659611107124912 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9096649207146444, + "performance_delta": 0.005883619143287611, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8986151969577384, + "call_delta": 0.03436894597115592 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9070101169548683, + "performance_delta": -0.00014350290593390103, + "baseline_call": 0.98880677333716, + "candidate_call": 0.9903135538494654, + "call_delta": 0.0015067805123053502 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9068666140489345, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9901700509435316, + "call_delta": -0.0005022601707684871 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9069383655019014, + "performance_delta": -0.00021525435890079603, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9904570567553993, + "call_delta": 0.0032288153835114963 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9027767812298199, + "performance_delta": 0.0027983066657099043, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9891655306019946, + "call_delta": 0.027480806486331333 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9030637870416875, + "performance_delta": 0.0030135610246107003, + "baseline_call": 0.969433881036091, + "candidate_call": 0.988017507354524, + "call_delta": 0.018583626318433022 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.9031355384946546, + "performance_delta": 0.0027983066657100153, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9648417880462079, + "call_delta": -0.005668364784386926 + } + ], + "rule": "historical" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_0.5", + "counts": { + "total": 48, + "performance_wins": 16, + "performance_ties": 1, + "performance_losses": 31, + "call_less": 4, + "call_ties": 0, + "call_more": 44, + "both_performance_win_and_call_less": 1, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22427095268610697, + "performance_delta": -0.0054665657031950365, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.9853528628495339, + "call_delta": 0.19474034620505987 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.22607173521804688, + "performance_delta": -0.00638261830049322, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.9833555259653795, + "call_delta": 0.20239680426098527 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.2255853947574774, + "performance_delta": -0.0071831204922556124, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.9810252996005326, + "call_delta": 0.19973368841544614 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.27125656976472395, + "performance_delta": 0.0029331642554513704, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.9856857523302264, + "call_delta": 0.16944074567243672 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.27047490574776584, + "performance_delta": 0.004547867977237496, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.9846870838881492, + "call_delta": 0.1454727030625833 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.2711912470021881, + "performance_delta": 0.0028506889131397206, + "baseline_call": 0.829560585885486, + "candidate_call": 0.9866844207723036, + "call_delta": 0.15712383488681758 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.26009300196494534, + "performance_delta": -0.007240694925530977, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.5699067909454061, + "call_delta": 0.19806924101198398 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2610246960878656, + "performance_delta": 0.002412661221895207, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.5223035952063915, + "call_delta": 0.16278295605858856 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2584417288296328, + "performance_delta": -0.007127772095523566, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5529294274300932, + "call_delta": 0.0985352862849534 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.2741835645844183, + "performance_delta": -0.023586121016129113, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.6231691078561917, + "call_delta": 0.1321571238348868 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2747498073529785, + "performance_delta": -0.02463923555262848, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.6195073235685752, + "call_delta": 0.16511318242343537 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2746117403663106, + "performance_delta": -0.02835996182435252, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.6088548601864181, + "call_delta": 0.15412782956058585 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.8068691750086296, + "performance_delta": -0.0050051777701070055, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.08715913013462202, + "call_delta": 0.04504659993096306 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.8068691750086296, + "performance_delta": -0.0010355540214014303, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.09043838453572661, + "call_delta": 0.03503624439074905 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.8080773213669313, + "performance_delta": -0.007076285812909866, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.09354504659993096, + "call_delta": 0.04918881601656886 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.8225750776665516, + "performance_delta": 0.009147393855712727, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.09941318605453918, + "call_delta": 0.08163617535381429 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.8210217466344494, + "performance_delta": 0.00793924749741104, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.10027614773904038, + "call_delta": 0.08715913013462201 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8217121159820504, + "performance_delta": 0.00742147048671038, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.0949257852951329, + "call_delta": 0.08646876078702107 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.8843631342768381, + "performance_delta": -0.01484294097342076, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.08025543665861236, + "call_delta": 0.05384880911287539 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.8803935105281325, + "performance_delta": -0.01812219537452542, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.08940283051432517, + "call_delta": 0.06541249568519159 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8793579565067311, + "performance_delta": -0.020365895754228536, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.09199171556782879, + "call_delta": 0.06989989644459786 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.8936831204694512, + "performance_delta": -0.005868139454608179, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.08232654470141526, + "call_delta": 0.0629962029685882 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.8929927511218502, + "performance_delta": -0.00793924749741115, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.08111839834311356, + "call_delta": 0.06420434932688988 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.8933379357956507, + "performance_delta": -0.006903693476009609, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.08232654470141526, + "call_delta": 0.06679323438039352 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.5032753878002438, + "performance_delta": 0.04667782094546685, + "baseline_call": 0.407140628987891, + "candidate_call": 0.9798088648158126, + "call_delta": 0.5726682358279216 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.5033321044721096, + "performance_delta": 0.04015540368091208, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.9834954484870828, + "call_delta": 0.5278904233899555 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5037574795111023, + "performance_delta": 0.05742563026401609, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.9825879817372316, + "call_delta": 0.5784533363582225 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7408048095737743, + "performance_delta": -0.008535859115787026, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.9809431982531265, + "call_delta": 0.12199756118310978 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7410033179253042, + "performance_delta": 0.028386694268780377, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.9824178317216345, + "call_delta": 0.29390579360803104 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7419391430110881, + "performance_delta": 0.08748546635283438, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9827581317528288, + "call_delta": 0.4203839718685308 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6695119530385957, + "performance_delta": -0.005104500467912554, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.019680685137396136, + "call_delta": -0.21316961120721437 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6695403113745285, + "performance_delta": -0.001673141820037971, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.019935910160791766, + "call_delta": -0.2569832402234637 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.669625386382327, + "performance_delta": -0.0043671837336585195, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.016929926551909932, + "call_delta": -0.25465785667697016 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7471287184867992, + "performance_delta": -0.011428409380937588, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.9701103139267788, + "call_delta": 0.40257493690270263 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7488018603068372, + "performance_delta": -0.009556759209369603, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.959107279584834, + "call_delta": 0.3492612653489493 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7490287269943, + "performance_delta": -0.01137169270907179, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9377818109633327, + "call_delta": 0.3356776224371154 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9066513596900337, + "performance_delta": -0.0014350290593384551, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.987012987012987, + "call_delta": 0.06385879314056109 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9075123771256368, + "performance_delta": -0.0005022601707684871, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9893807849608954, + "call_delta": 0.05503336442562956 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9069383655019014, + "performance_delta": -0.0006457630767022771, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9877305015426563, + "call_delta": 0.11946616918992614 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080863887493722, + "performance_delta": -7.1751452966895e-05, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.9687163665064218, + "call_delta": 0.14142211379780445 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9065796082370668, + "performance_delta": -0.00043050871780148103, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9649852909521418, + "call_delta": 0.11473057329410918 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9091626605438761, + "performance_delta": 0.0053813589725192346, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9574513883906149, + "call_delta": 0.09320513740403247 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9070101169548683, + "performance_delta": -0.00014350290593390103, + "baseline_call": 0.98880677333716, + "candidate_call": 0.990744062567267, + "call_delta": 0.0019372892301069422 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9070101169548683, + "performance_delta": 0.0, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9913898256439693, + "call_delta": 0.0007175145296691721 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9070818684078352, + "performance_delta": -7.175145296700602e-05, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9910310683791347, + "call_delta": 0.0038028270072468784 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9029202841357538, + "performance_delta": 0.0029418095716438053, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9893807849608954, + "call_delta": 0.02769606084523213 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9034942957594891, + "performance_delta": 0.0034440697424122924, + "baseline_call": 0.969433881036091, + "candidate_call": 0.9801965989811293, + "call_delta": 0.010762717945038358 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.9028485326827869, + "performance_delta": 0.0025113008538423243, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9671378345411494, + "call_delta": -0.0033723182894453974 + } + ], + "rule": "historical" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_0.75", + "counts": { + "total": 48, + "performance_wins": 23, + "performance_ties": 0, + "performance_losses": 25, + "call_less": 10, + "call_ties": 0, + "call_more": 38, + "both_performance_win_and_call_less": 7, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.2319989174313225, + "performance_delta": 0.0022613990420204955, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7762982689747004, + "call_delta": -0.014314247669773628 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23674301754245702, + "performance_delta": 0.004288664023916922, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7596537949400799, + "call_delta": -0.02130492676431428 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.23714462398254227, + "performance_delta": 0.004376108732809247, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.8578561917443409, + "call_delta": 0.07656458055925441 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2698761268543865, + "performance_delta": 0.0015527213451139166, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8032623169107856, + "call_delta": -0.012982689747004006 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2684151526427204, + "performance_delta": 0.002488114872192082, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8169107856191744, + "call_delta": -0.02230359520639147 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.2696999750942859, + "performance_delta": 0.0013594170052375065, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8199067909454061, + "call_delta": -0.009653794940079896 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.26372894434686617, + "performance_delta": -0.0036047525436101457, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.3848202396804261, + "call_delta": 0.012982689747004006 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2621221451028754, + "performance_delta": 0.0035101102369050063, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.3725033288948069, + "call_delta": 0.012982689747004006 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.25716044964994544, + "performance_delta": -0.00840905127521091, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.40346205059920104, + "call_delta": -0.05093209054593878 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.29493861915351943, + "performance_delta": -0.0028310664470279634, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.5512649800266312, + "call_delta": 0.06025299600532624 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2927064243318904, + "performance_delta": -0.006682618573716548, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5825565912117177, + "call_delta": 0.12816245006657784 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2929922062010103, + "performance_delta": -0.009979495989652809, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.5808921438082557, + "call_delta": 0.12616511318242346 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.8120469451156369, + "performance_delta": 0.00017259233690025688, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.06247842595788747, + "call_delta": 0.020365895754228508 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.810666206420435, + "performance_delta": 0.002761477390403888, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.06644804970659303, + "call_delta": 0.011045909561615469 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.8137728684846393, + "performance_delta": -0.001380738695201944, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.06247842595788747, + "call_delta": 0.01812219537452537 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.8198136002761477, + "performance_delta": 0.0063859164653088385, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.07611322057300655, + "call_delta": 0.058336209872281664 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.8196410079392475, + "performance_delta": 0.006558508802209206, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.08836727649292371, + "call_delta": 0.07525025888850534 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8203313772868485, + "performance_delta": 0.006040731791508436, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.09837763203313774, + "call_delta": 0.0899206075250259 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.8945460821539524, + "performance_delta": -0.004659993096306492, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.050914739385571284, + "call_delta": 0.024508111839834316 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.8924749741111495, + "performance_delta": -0.006040731791508436, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.06127027959958578, + "call_delta": 0.0372799447704522 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8988608905764585, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.054366586123576116, + "call_delta": 0.03227476700034519 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.8976527442181567, + "performance_delta": -0.0018985157059027147, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.07611322057300655, + "call_delta": 0.05678287884017949 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.8966171901967552, + "performance_delta": -0.004314808422506089, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.07076285812909906, + "call_delta": 0.05384880911287539 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.897134967207456, + "performance_delta": -0.003106662064204291, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.06989989644459786, + "call_delta": 0.054366586123576116 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48044692737430167, + "performance_delta": 0.023849360519524687, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5296202818818592, + "call_delta": 0.12247965289396817 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.4890111448260216, + "performance_delta": 0.025834444034824045, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.550860675495562, + "call_delta": 0.09525565039843464 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.4808723024132944, + "performance_delta": 0.034540453166208185, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5036156878314381, + "call_delta": 0.09948104245242889 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7464481184244108, + "performance_delta": -0.002892550265150451, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.9825879817372316, + "call_delta": 0.12364234466721491 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7467033434478065, + "performance_delta": 0.03408671979128275, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.9870402404786887, + "call_delta": 0.29852820236508515 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7468734934634036, + "performance_delta": 0.09241981680514988, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9880611405722712, + "call_delta": 0.4256869806879733 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.671865694921022, + "performance_delta": -0.002750758585486235, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.02064486855911295, + "call_delta": -0.21220542778549753 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6721492782803505, + "performance_delta": 0.0009358250857840478, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.018659785043813628, + "call_delta": -0.25825936534044186 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6705612114681111, + "performance_delta": -0.0034313586478744718, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.016306043161387288, + "call_delta": -0.2552817400674928 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7471287184867992, + "performance_delta": -0.011428409380937588, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.9700252389189802, + "call_delta": 0.402489861894904 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7486600686271729, + "performance_delta": -0.009698550889033819, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9587386212177069, + "call_delta": 0.34889260698182223 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7493690270254941, + "performance_delta": -0.011031392677877672, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9490117119927403, + "call_delta": 0.346907523466523 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9062926024251992, + "performance_delta": -0.0017937863241730412, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.987945755901557, + "call_delta": 0.06479156202913106 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9066513596900337, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9885915189782594, + "call_delta": 0.054244098442993494 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.00021525435890079603, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9881610102604578, + "call_delta": 0.11989667790772762 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9084451460142068, + "performance_delta": 0.00028700581186769103, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.9588146659969864, + "call_delta": 0.13152041328836905 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9062926024251992, + "performance_delta": -0.0007175145296691721, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.966563822917414, + "call_delta": 0.11630910525938143 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9085168974671737, + "performance_delta": 0.0047355958958168465, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9580254000143503, + "call_delta": 0.09377914902776785 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9069383655019014, + "performance_delta": -0.00021525435890079603, + "baseline_call": 0.98880677333716, + "candidate_call": 0.9915333285499032, + "call_delta": 0.0027265552127431203 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9067231111430006, + "performance_delta": -0.00028700581186769103, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9921073401736386, + "call_delta": 0.0014350290593384551 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9068666140489345, + "performance_delta": -0.00028700581186769103, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9906723111143001, + "call_delta": 0.0034440697424122924 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9031355384946546, + "performance_delta": 0.0031570639305446013, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9893090335079285, + "call_delta": 0.027624309392265234 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9039965559302576, + "performance_delta": 0.003946329913180779, + "baseline_call": 0.969433881036091, + "candidate_call": 0.9791920786395925, + "call_delta": 0.009758197603501495 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.902705029776853, + "performance_delta": 0.0023677979479084232, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9555858506134749, + "call_delta": -0.014924302217119934 + } + ], + "rule": "historical" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_1", + "counts": { + "total": 48, + "performance_wins": 26, + "performance_ties": 2, + "performance_losses": 20, + "call_less": 24, + "call_ties": 1, + "call_more": 23, + "both_performance_win_and_call_less": 16, + "both_performance_tie_and_call_less": 1 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22850906574549645, + "performance_delta": -0.0012284526438055554, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7906125166444741, + "call_delta": 0.0 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23004315338506715, + "performance_delta": -0.002411200133472957, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7926098535286284, + "call_delta": 0.011651131824234273 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.2308450652014655, + "performance_delta": -0.001923450048267522, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.7766311584553928, + "call_delta": -0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2678997047385957, + "performance_delta": -0.0004237007706768825, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8209054593874834, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2672954581839896, + "performance_delta": 0.0013684204134612465, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8215712383488681, + "call_delta": -0.017643142476697737 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.267918819975632, + "performance_delta": -0.0004217381134163922, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8149134487350199, + "call_delta": -0.01464713715046606 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2675187098549581, + "performance_delta": 0.00018501296448181215, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.4144474034620506, + "call_delta": 0.042609853528628505 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2578686516978949, + "performance_delta": -0.0007433831680754843, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.35918774966711053, + "call_delta": -0.00033288948069237767 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.26086905351963807, + "performance_delta": -0.004700447405518282, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.39980026631158455, + "call_delta": -0.05459387483355527 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.3029107097560361, + "performance_delta": 0.005141024155488727, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.4337549933422104, + "call_delta": -0.05725699067909451 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.29937410202312603, + "performance_delta": -1.4940882480940498e-05, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.44274300932090543, + "call_delta": -0.011651131824234384 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3009613147999525, + "performance_delta": -0.002010387390710633, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.4477363515312916, + "call_delta": -0.006990679094540653 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.8125647221263376, + "performance_delta": 0.0006903693476009165, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.017604418363824648, + "call_delta": -0.024508111839834312 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.8134276838108389, + "performance_delta": 0.005522954780807776, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.010700724887814981, + "call_delta": -0.04470141525716258 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.81429064549534, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.006558508802209182, + "call_delta": -0.03779772178115291 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0008629616845011734, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.007766655160510874, + "call_delta": -0.010010355540214014 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0012081463583016872, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.005695547117707974, + "call_delta": -0.00742147048671039 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.00638591646530894, + "call_delta": -0.0020711080428028996 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.8998964445978599, + "performance_delta": 0.0006903693476010275, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.027614773904038662, + "call_delta": 0.001208146358301694 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.901449775629962, + "performance_delta": 0.002934069727304034, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.010873317224715223, + "call_delta": -0.013117017604418364 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.901449775629962, + "performance_delta": 0.0017259233690023468, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.010355540214014497, + "call_delta": -0.01173627890921643 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0020711080428028605, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.01156368657231619, + "call_delta": -0.0077666551605108745 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0006903693476009165, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.006731101139109423, + "call_delta": -0.010182947877114256 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.001380738695201944, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.007076285812909907, + "call_delta": -0.008457024508111841 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48586336953747555, + "performance_delta": 0.029265802682698572, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5045798712531548, + "call_delta": 0.09743924226526385 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.49794402064486853, + "performance_delta": 0.034767319853670986, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.5430054164421632, + "call_delta": 0.08740039134503585 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.49649774551229336, + "performance_delta": 0.05016589626520718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5379576326461163, + "call_delta": 0.13382298726710712 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7511272438533307, + "performance_delta": 0.001786575163769455, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.852763519836656, + "call_delta": -0.006182117233360707 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7536511357513541, + "performance_delta": 0.04103451209483033, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8933726568924936, + "call_delta": 0.20486061877889006 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7511839605251964, + "performance_delta": 0.09673028386694271, + "baseline_call": 0.562374159884298, + "candidate_call": 0.8652411876471089, + "call_delta": 0.30286702776281094 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6800045373337492, + "performance_delta": 0.005388083827240986, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.12287666959702805, + "call_delta": -0.10997362674758245 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6805717040524062, + "performance_delta": 0.0093582508578397, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.09480191702350906, + "call_delta": -0.18211723336074642 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6810254374273318, + "performance_delta": 0.007032867311346225, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.12109009443325866, + "call_delta": -0.15049768879562145 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7519496355953833, + "performance_delta": -0.006607492272353466, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.8643620792331906, + "call_delta": 0.29682670220911445 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7473555851742619, + "performance_delta": -0.011003034341944828, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9705640473017043, + "call_delta": 0.3607180330658196 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7550406942120637, + "performance_delta": -0.005359725491308143, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9094518333664181, + "call_delta": 0.30734764484020083 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9060773480662984, + "performance_delta": -0.002009040683073837, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9890937791490277, + "call_delta": 0.06593958527660182 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0015785319652723562, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9886632704312263, + "call_delta": 0.05431584989596039 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9071536198608022, + "performance_delta": -0.00043050871780148103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.98973954222573, + "call_delta": 0.12147520987299987 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9052163306306953, + "performance_delta": -0.0017937863241730412, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9674248403530171, + "call_delta": 0.1171701226949845 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9075123771256368, + "performance_delta": 0.0037310755542799834, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9586711630910526, + "call_delta": 0.09442491210447013 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9071536198608022, + "performance_delta": 0.0, + "baseline_call": 0.98880677333716, + "candidate_call": 0.99253784889144, + "call_delta": 0.0037310755542799834 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9066513596900337, + "performance_delta": -0.00035875726483458603, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9929683576092416, + "call_delta": 0.0022960464949415282 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.989811293678697, + "call_delta": 0.0025830523068092193 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9034225443065222, + "performance_delta": 0.0034440697424122924, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9885197675252924, + "call_delta": 0.026835043409629056 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9037095501183899, + "performance_delta": 0.0036593241013130884, + "baseline_call": 0.969433881036091, + "candidate_call": 0.9783310612039894, + "call_delta": 0.008897180167898422 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.9026332783238861, + "performance_delta": 0.0022960464949415282, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9682141063356533, + "call_delta": -0.0022960464949415282 + } + ], + "rule": "historical" + }, + { + "rule": "historical", + "baseline": "original", + "candidate": "confidence", + "counts": { + "total": 48, + "performance_wins": 12, + "performance_ties": 0, + "performance_losses": 36, + "call_less": 12, + "call_ties": 0, + "call_more": 36, + "both_performance_win_and_call_less": 3, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.799275112185019, + "performance_delta": -0.01587849499482219, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.113565757680359, + "call_delta": 0.0692095270969969 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.799275112185019, + "performance_delta": -0.01587849499482219, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.113565757680359, + "call_delta": 0.0692095270969969 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.799275112185019, + "performance_delta": -0.01587849499482219, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.113565757680359, + "call_delta": 0.0692095270969969 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8158439765274422, + "performance_delta": 0.001553331032102201, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.113565757680359, + "call_delta": 0.10510873317224716 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8158439765274422, + "performance_delta": 0.001553331032102201, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.113565757680359, + "call_delta": 0.10510873317224716 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8158439765274422, + "performance_delta": 0.001553331032102201, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.113565757680359, + "call_delta": 0.10510873317224716 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8769416637901277, + "performance_delta": -0.02278218847083191, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.08698653779772178, + "call_delta": 0.06489471867449086 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8769416637901277, + "performance_delta": -0.02278218847083191, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.08698653779772178, + "call_delta": 0.06489471867449086 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8769416637901277, + "performance_delta": -0.02278218847083191, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.08698653779772178, + "call_delta": 0.06489471867449086 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.8878149810148429, + "performance_delta": -0.012426648256817385, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.08698653779772178, + "call_delta": 0.07145322747670003 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.8878149810148429, + "performance_delta": -0.012426648256817385, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.08698653779772178, + "call_delta": 0.07145322747670003 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.8878149810148429, + "performance_delta": -0.012426648256817385, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.08698653779772178, + "call_delta": 0.07145322747670003 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.4968380455434875, + "performance_delta": 0.050506196296401296, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.9873805405098829, + "call_delta": 0.5832458951308737 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.4968380455434875, + "performance_delta": 0.050506196296401296, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.9873805405098829, + "call_delta": 0.5832458951308737 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.4968380455434875, + "performance_delta": 0.050506196296401296, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.9873805405098829, + "call_delta": 0.5832458951308737 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7387062927147435, + "performance_delta": 0.08425261605648982, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9873805405098829, + "call_delta": 0.42500638062558493 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7387062927147435, + "performance_delta": 0.08425261605648982, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9873805405098829, + "call_delta": 0.42500638062558493 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7387062927147435, + "performance_delta": 0.08425261605648982, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9873805405098829, + "call_delta": 0.42500638062558493 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6187788900547316, + "performance_delta": -0.055213680061253934, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.5209426310864078, + "call_delta": 0.2493548478575277 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6187788900547316, + "performance_delta": -0.055213680061253934, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.5209426310864078, + "call_delta": 0.2493548478575277 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6187788900547316, + "performance_delta": -0.055213680061253934, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.5209426310864078, + "call_delta": 0.2493548478575277 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7243286163967898, + "performance_delta": -0.036071803306581995, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6838612710206166, + "call_delta": 0.08175708249439928 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7243286163967898, + "performance_delta": -0.036071803306581995, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6838612710206166, + "call_delta": 0.08175708249439928 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7243286163967898, + "performance_delta": -0.036071803306581995, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6838612710206166, + "call_delta": 0.08175708249439928 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9077276314845376, + "performance_delta": 0.00014350290593390103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8580038745784602, + "call_delta": -0.010260457774269982 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9077276314845376, + "performance_delta": 0.00014350290593390103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8580038745784602, + "call_delta": -0.010260457774269982 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9077276314845376, + "performance_delta": 0.00014350290593390103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8580038745784602, + "call_delta": -0.010260457774269982 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9024180239649853, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8580038745784602, + "call_delta": -0.006242376408122308 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9024180239649853, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8580038745784602, + "call_delta": -0.006242376408122308 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9024180239649853, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8580038745784602, + "call_delta": -0.006242376408122308 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.21609512872414097, + "performance_delta": -0.016673386525592054, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.9790279627163782, + "call_delta": 0.19773635153129165 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.21609512872414097, + "performance_delta": -0.016673386525592054, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.9790279627163782, + "call_delta": 0.19773635153129165 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.21609512872414097, + "performance_delta": -0.016673386525592054, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.9790279627163782, + "call_delta": 0.19773635153129165 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.26818431283513144, + "performance_delta": -0.00015624525391694233, + "baseline_call": 0.829560585885486, + "candidate_call": 0.9860186418109188, + "call_delta": 0.15645805592543283 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.26818431283513144, + "performance_delta": -0.00015624525391694233, + "baseline_call": 0.829560585885486, + "candidate_call": 0.9860186418109188, + "call_delta": 0.15645805592543283 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.26818431283513144, + "performance_delta": -0.00015624525391694233, + "baseline_call": 0.829560585885486, + "candidate_call": 0.9860186418109188, + "call_delta": 0.15645805592543283 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9593169261677549, + "call_delta": -0.027911315204132925 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9593169261677549, + "call_delta": -0.027911315204132925 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9593169261677549, + "call_delta": -0.027911315204132925 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.8996197172992753, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9593169261677549, + "call_delta": -0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.8996197172992753, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9593169261677549, + "call_delta": -0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.8996197172992753, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9593169261677549, + "call_delta": -0.01119322666283995 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2523047077304935, + "performance_delta": -0.013264793194662838, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.9254327563249002, + "call_delta": 0.47103861517976037 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2523047077304935, + "performance_delta": -0.013264793194662838, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.9254327563249002, + "call_delta": 0.47103861517976037 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2523047077304935, + "performance_delta": -0.013264793194662838, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.9254327563249002, + "call_delta": 0.47103861517976037 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2597490004762105, + "performance_delta": -0.04322270171445258, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.9221038615179761, + "call_delta": 0.4673768308921438 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2597490004762105, + "performance_delta": -0.04322270171445258, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.9221038615179761, + "call_delta": 0.4673768308921438 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2597490004762105, + "performance_delta": -0.04322270171445258, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.9221038615179761, + "call_delta": 0.4673768308921438 + } + ] + }, + { + "baseline": "original", + "candidate": "score_gain", + "counts": { + "total": 48, + "performance_wins": 22, + "performance_ties": 12, + "performance_losses": 14, + "call_less": 12, + "call_ties": 12, + "call_more": 24, + "both_performance_win_and_call_less": 12, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.23610378506203042, + "performance_delta": 0.00636626667272841, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7952729693741678, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23330554571941767, + "performance_delta": 0.0008511922008775652, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7902796271637816, + "call_delta": 0.009320905459387463 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.22951409007632145, + "performance_delta": -0.003254425173411568, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.8009320905459387, + "call_delta": 0.019640479360852225 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2687941772737477, + "performance_delta": 0.0004707717644751308, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8062583222370173, + "call_delta": -0.00998668442077233 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2699197889196931, + "performance_delta": 0.003992751149164753, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.831225033288948, + "call_delta": -0.007989347536617841 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.2687146609089476, + "performance_delta": 0.00037410281989919136, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8225699067909454, + "call_delta": -0.006990679094540542 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2668710925054341, + "performance_delta": -0.0004626043850421979, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.39580559254327563, + "call_delta": 0.023968042609853524 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.25610142784250745, + "performance_delta": -0.002510607023462952, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.46804260985352863, + "call_delta": 0.10852197070572572 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2553143923177106, + "performance_delta": -0.010255108607445773, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5402796271637816, + "call_delta": 0.08588548601864182 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.3034889605344151, + "performance_delta": 0.005719274933867702, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.46937416777629826, + "call_delta": -0.021637816245006658 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2998980207837558, + "performance_delta": 0.000508977878148853, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.4284287616511318, + "call_delta": -0.025965379494008012 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3033337683996216, + "performance_delta": 0.0003620662089585114, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.45272969374167776, + "call_delta": -0.001997336884154488 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8134276838108389, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.0, + "candidate_call": 0.010700724887814981, + "call_delta": 0.010700724887814981 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.8998964445978599, + "performance_delta": -0.0017259233690023468, + "baseline_call": 0.0, + "candidate_call": 0.027614773904038662, + "call_delta": 0.027614773904038662 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48586336953747555, + "performance_delta": 0.029265802682698572, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5045798712531548, + "call_delta": 0.09743924226526385 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.49794402064486853, + "performance_delta": 0.034767319853670986, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.5430054164421632, + "call_delta": 0.08740039134503585 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.49649774551229336, + "performance_delta": 0.05016589626520718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5379576326461163, + "call_delta": 0.13382298726710712 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7514108272126592, + "performance_delta": 0.002070158523097887, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.8495023112043786, + "call_delta": -0.00944332586563812 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.752006352267249, + "performance_delta": 0.039389728610725205, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8824263392224144, + "call_delta": 0.19391430110881092 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7502197771034795, + "performance_delta": 0.09576610044522582, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9653177551541275, + "call_delta": 0.4029435952698296 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6800045373337492, + "performance_delta": 0.005388083827240986, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.12287666959702805, + "call_delta": -0.10997362674758245 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6805717040524062, + "performance_delta": 0.0093582508578397, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.09480191702350906, + "call_delta": -0.18211723336074642 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6810254374273318, + "performance_delta": 0.007032867311346225, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.12109009443325866, + "call_delta": -0.15049768879562145 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7447182599325072, + "performance_delta": -0.013838867935229593, + "baseline_call": 0.5675353770240762, + "candidate_call": 1.0, + "call_delta": 0.4324646229759238 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7473555851742619, + "performance_delta": -0.011003034341944828, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.968380455434875, + "call_delta": 0.35853444119899036 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7482063352522474, + "performance_delta": -0.012194084451124354, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.96633865524771, + "call_delta": 0.36423446672149273 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9231541938724259, + "candidate_call": 1.0, + "call_delta": 0.07684580612757408 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.0006457630767023881, + "baseline_call": 0.9343474205352659, + "candidate_call": 1.0, + "call_delta": 0.06565257946473413 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.00021525435890079603, + "baseline_call": 0.8682643323527302, + "candidate_call": 1.0, + "call_delta": 0.13173566764726985 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9060055966133315, + "performance_delta": -0.001004520341536863, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9787615699217909, + "call_delta": 0.12850685226375824 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9049293248188276, + "performance_delta": 0.0011480232474707641, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9672813374470833, + "call_delta": 0.10303508646050086 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.000215254358900685, + "baseline_call": 0.98880677333716, + "candidate_call": 1.0, + "call_delta": 0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9039965559302576, + "performance_delta": 0.004018081366147674, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9788333213747579, + "call_delta": 0.017148597259094567 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9023462725120184, + "performance_delta": 0.0017220348712061462, + "baseline_call": 1.0, + "candidate_call": 0.9800530960751955, + "call_delta": -0.01994690392480447 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9036377986654229, + "performance_delta": 0.0030135610246107003, + "baseline_call": 1.0, + "candidate_call": 0.9882327617134247, + "call_delta": -0.011767238286575332 + } + ], + "rule": "endpoint" + }, + { + "baseline": "original", + "candidate": "checkpoint_perf", + "counts": { + "total": 48, + "performance_wins": 7, + "performance_ties": 18, + "performance_losses": 23, + "call_less": 17, + "call_ties": 17, + "call_more": 14, + "both_performance_win_and_call_less": 3, + "both_performance_tie_and_call_less": 1 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.2305421865004371, + "performance_delta": 0.0008046681111350817, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7816245006657789, + "call_delta": -0.00898801597869514 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.22838091389772372, + "performance_delta": -0.0040734396208163826, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.8049267643142477, + "call_delta": 0.023968042609853524 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.23035654564059277, + "performance_delta": -0.002411969609140252, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.774300932090546, + "call_delta": -0.006990679094540542 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.26792218076116586, + "performance_delta": -0.00040122474810672104, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8215712383488681, + "call_delta": 0.005326231691078487 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.26876666264349774, + "performance_delta": 0.002839624872969404, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8125832223701731, + "call_delta": -0.026631158455392767 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.26833406705021806, + "performance_delta": -6.491038830325202e-06, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8229027962716379, + "call_delta": -0.0066577896138481085 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2624005680779394, + "performance_delta": -0.0049331288125369355, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.37350199733688416, + "call_delta": 0.0016644474034620549 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.25991507502469235, + "performance_delta": 0.0013030401587219576, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.37283621837549935, + "call_delta": 0.013315579227696439 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2613858995371947, + "performance_delta": -0.004183601387961644, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.3808255659121172, + "call_delta": -0.07356857523302263 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.29875083466112995, + "performance_delta": 0.0009811490605825535, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.43275632490013316, + "call_delta": -0.058255659121171754 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.29645395142312325, + "performance_delta": -0.0029350914824837204, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.524300932090546, + "call_delta": 0.06990679094540614 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3014472023134628, + "performance_delta": -0.0015244998772003249, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.4490679094540613, + "call_delta": -0.0056591211717709755 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.8967897825336555, + "performance_delta": -0.004832585433206749, + "baseline_call": 0.0, + "candidate_call": 0.038488091128753885, + "call_delta": 0.038488091128753885 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.8979979288919572, + "performance_delta": -0.0036244390749050615, + "baseline_call": 0.0, + "candidate_call": 0.0277873662409389, + "call_delta": 0.0277873662409389 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.445622890848765, + "performance_delta": -0.010974676006011985, + "baseline_call": 0.407140628987891, + "candidate_call": 0.4267929557893543, + "call_delta": 0.019652326801463282 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.4702379264384766, + "performance_delta": 0.007061225647279068, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.47905736891359213, + "call_delta": 0.023452343816464827 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.44219153220089047, + "performance_delta": -0.004140317046195718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.4057227121912486, + "call_delta": 0.0015880668122394415 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.6876612880356181, + "performance_delta": -0.06167938065394318, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.6266057907721975, + "call_delta": -0.23233984629781923 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.6622805773757195, + "performance_delta": -0.050336046280804236, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.5790488614128123, + "call_delta": -0.1094631767007912 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.6775940787794572, + "performance_delta": 0.023140402121203496, + "baseline_call": 0.562374159884298, + "candidate_call": 0.6149505147037971, + "call_delta": 0.05257635481949918 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6657119360235941, + "performance_delta": -0.008904517482914098, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.322916371267334, + "call_delta": 0.09006607492272353 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6712134531945665, + "performance_delta": 0.0, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.27691915038425546, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6739925701159856, + "performance_delta": 0.0, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.2715877832288801, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7582735445084082, + "performance_delta": -0.00028358335932854306, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.537475540935258, + "call_delta": -0.030059836088818237 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7583586195162068, + "performance_delta": 0.0, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.6098460142358847, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7604004197033718, + "performance_delta": 0.0, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6021041885262173, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9080863887493722, + "performance_delta": 0.0, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9231541938724259, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9079428858434383, + "performance_delta": -7.175145296700602e-05, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9026332783238861, + "call_delta": -0.0317141422113798 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.907297122766736, + "performance_delta": -0.00028700581186769103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8648202626103179, + "call_delta": -0.0034440697424122924 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9072253713137691, + "performance_delta": -0.0009327688885699681, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.8425773121905719, + "call_delta": 0.01528305948195452 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9070101169548683, + "performance_delta": 0.0, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.8502547176580326, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9037813015713568, + "performance_delta": 0.0, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8642462509865825, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9071536198608022, + "performance_delta": 0.0, + "baseline_call": 0.98880677333716, + "candidate_call": 0.9778288010332209, + "call_delta": -0.010977972303939154 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9070818684078352, + "performance_delta": -0.00028700581186769103, + "baseline_call": 1.0, + "candidate_call": 0.9764655234268494, + "call_delta": -0.023534476573150553 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9047858219128937, + "performance_delta": -0.0025830523068092193, + "baseline_call": 1.0, + "candidate_call": 0.9235129511372605, + "call_delta": -0.07648704886273949 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9013417521704814, + "performance_delta": 0.0013632776063714491, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9797660902633278, + "call_delta": 0.018081366147664535 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9002654803759776, + "performance_delta": -0.00035875726483458603, + "baseline_call": 1.0, + "candidate_call": 0.966563822917414, + "call_delta": -0.03343617708258595 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9003372318289445, + "performance_delta": -0.00028700581186769103, + "baseline_call": 1.0, + "candidate_call": 0.9903853053024324, + "call_delta": -0.009614694697567594 + } + ], + "rule": "endpoint" + }, + { + "baseline": "original", + "candidate": "both", + "counts": { + "total": 48, + "performance_wins": 17, + "performance_ties": 10, + "performance_losses": 21, + "call_less": 14, + "call_ties": 11, + "call_more": 23, + "both_performance_win_and_call_less": 8, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22850906574549645, + "performance_delta": -0.0012284526438055554, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7906125166444741, + "call_delta": 0.0 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23004315338506715, + "performance_delta": -0.002411200133472957, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7926098535286284, + "call_delta": 0.011651131824234273 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.2308450652014655, + "performance_delta": -0.001923450048267522, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.7766311584553928, + "call_delta": -0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2678997047385957, + "performance_delta": -0.0004237007706768825, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8209054593874834, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2672954581839896, + "performance_delta": 0.0013684204134612465, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8215712383488681, + "call_delta": -0.017643142476697737 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.267918819975632, + "performance_delta": -0.0004217381134163922, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8149134487350199, + "call_delta": -0.01464713715046606 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2675187098549581, + "performance_delta": 0.00018501296448181215, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.4144474034620506, + "call_delta": 0.042609853528628505 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2578686516978949, + "performance_delta": -0.0007433831680754843, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.35918774966711053, + "call_delta": -0.00033288948069237767 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.26086905351963807, + "performance_delta": -0.004700447405518282, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.39980026631158455, + "call_delta": -0.05459387483355527 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.3029107097560361, + "performance_delta": 0.005141024155488727, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.4337549933422104, + "call_delta": -0.05725699067909451 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.29937410202312603, + "performance_delta": -1.4940882480940498e-05, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.44274300932090543, + "call_delta": -0.011651131824234384 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3009613147999525, + "performance_delta": -0.002010387390710633, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.4477363515312916, + "call_delta": -0.006990679094540653 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8125647221263376, + "performance_delta": -0.0017259233690024578, + "baseline_call": 0.0, + "candidate_call": 0.017604418363824648, + "call_delta": 0.017604418363824648 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8134276838108389, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.0, + "candidate_call": 0.010700724887814981, + "call_delta": 0.010700724887814981 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.8998964445978599, + "performance_delta": -0.0017259233690023468, + "baseline_call": 0.0, + "candidate_call": 0.027614773904038662, + "call_delta": 0.027614773904038662 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.901449775629962, + "performance_delta": -0.00017259233690025688, + "baseline_call": 0.0, + "candidate_call": 0.010873317224715223, + "call_delta": 0.010873317224715223 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48586336953747555, + "performance_delta": 0.029265802682698572, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5045798712531548, + "call_delta": 0.09743924226526385 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.49794402064486853, + "performance_delta": 0.034767319853670986, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.5430054164421632, + "call_delta": 0.08740039134503585 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.49649774551229336, + "performance_delta": 0.05016589626520718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5379576326461163, + "call_delta": 0.13382298726710712 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7511272438533307, + "performance_delta": 0.001786575163769455, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.852763519836656, + "call_delta": -0.006182117233360707 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7536511357513541, + "performance_delta": 0.04103451209483033, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8933726568924936, + "call_delta": 0.20486061877889006 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7511839605251964, + "performance_delta": 0.09673028386694271, + "baseline_call": 0.562374159884298, + "candidate_call": 0.8652411876471089, + "call_delta": 0.30286702776281094 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6800045373337492, + "performance_delta": 0.005388083827240986, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.12287666959702805, + "call_delta": -0.10997362674758245 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6805717040524062, + "performance_delta": 0.0093582508578397, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.09480191702350906, + "call_delta": -0.18211723336074642 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6810254374273318, + "performance_delta": 0.007032867311346225, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.12109009443325866, + "call_delta": -0.15049768879562145 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7519496355953833, + "performance_delta": -0.006607492272353466, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.8643620792331906, + "call_delta": 0.29682670220911445 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7473555851742619, + "performance_delta": -0.011003034341944828, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9705640473017043, + "call_delta": 0.3607180330658196 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7550406942120637, + "performance_delta": -0.005359725491308143, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9094518333664181, + "call_delta": 0.30734764484020083 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9060773480662984, + "performance_delta": -0.002009040683073837, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9890937791490277, + "call_delta": 0.06593958527660182 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0015785319652723562, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9886632704312263, + "call_delta": 0.05431584989596039 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9071536198608022, + "performance_delta": -0.00043050871780148103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.98973954222573, + "call_delta": 0.12147520987299987 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9052163306306953, + "performance_delta": -0.0017937863241730412, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9674248403530171, + "call_delta": 0.1171701226949845 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9075123771256368, + "performance_delta": 0.0037310755542799834, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9586711630910526, + "call_delta": 0.09442491210447013 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.000215254358900685, + "baseline_call": 0.98880677333716, + "candidate_call": 1.0, + "call_delta": 0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9034225443065222, + "performance_delta": 0.0034440697424122924, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9885197675252924, + "call_delta": 0.026835043409629056 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9037095501183899, + "performance_delta": 0.0030853124775777063, + "baseline_call": 1.0, + "candidate_call": 0.9783310612039894, + "call_delta": -0.021668938796010617 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9026332783238861, + "performance_delta": 0.002009040683073837, + "baseline_call": 1.0, + "candidate_call": 0.9682141063356533, + "call_delta": -0.0317858936643467 + } + ], + "rule": "endpoint" + }, + { + "baseline": "original", + "candidate": "score_only_gain", + "counts": { + "total": 48, + "performance_wins": 18, + "performance_ties": 13, + "performance_losses": 17, + "call_less": 12, + "call_ties": 13, + "call_more": 23, + "both_performance_win_and_call_less": 9, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22725460966786967, + "performance_delta": -0.0024829087214323364, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.8768308921438083, + "call_delta": 0.0862183754993342 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23264549293538322, + "performance_delta": 0.0001911394168431224, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7882822902796272, + "call_delta": 0.007323568575232975 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.229420185942548, + "performance_delta": -0.0033483293071850107, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.7959387483355526, + "call_delta": 0.01464713715046606 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2678997047385957, + "performance_delta": -0.0004237007706768825, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8209054593874834, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.270279514537055, + "performance_delta": 0.0043524767665266695, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.801930758988016, + "call_delta": -0.03728362183754985 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.26989568494530464, + "performance_delta": 0.0015551268562562504, + "baseline_call": 0.829560585885486, + "candidate_call": 0.800266311584554, + "call_delta": -0.02929427430093201 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2668710925054341, + "performance_delta": -0.0004626043850421979, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.39580559254327563, + "call_delta": 0.023968042609853524 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2578686516978949, + "performance_delta": -0.0007433831680754843, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.35918774966711053, + "call_delta": -0.00033288948069237767 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2565642588848189, + "performance_delta": -0.00900524204033748, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.46571238348868177, + "call_delta": 0.011318242343541951 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.29895985090664157, + "performance_delta": 0.0011901653060941708, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.4497336884154461, + "call_delta": -0.04127829560585883 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2991984700035601, + "performance_delta": -0.0001905729020468816, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.453728362183755, + "call_delta": -0.0006657789613848109 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3033337683996216, + "performance_delta": 0.0003620662089585114, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.45272969374167776, + "call_delta": -0.001997336884154488 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9005868139454608, + "performance_delta": -0.0010355540214014303, + "baseline_call": 0.0, + "candidate_call": 0.021574042112530202, + "call_delta": 0.021574042112530202 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.511414230212971, + "performance_delta": 0.05481666335819407, + "baseline_call": 0.407140628987891, + "candidate_call": 0.7920199642684967, + "call_delta": 0.38487933528060575 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.503984346198565, + "performance_delta": 0.04080764540736748, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.597339988089499, + "call_delta": 0.14173496299237165 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5101097467600602, + "performance_delta": 0.06377789751297397, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.7076255565323427, + "call_delta": 0.30349091115333354 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7498794770722854, + "performance_delta": 0.0005388083827241319, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.846070952556504, + "call_delta": -0.012874684513512702 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7536511357513541, + "performance_delta": 0.04103451209483033, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8933726568924936, + "call_delta": 0.20486061877889006 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7510138105095993, + "performance_delta": 0.09656013385134554, + "baseline_call": 0.562374159884298, + "candidate_call": 0.8730964467005076, + "call_delta": 0.3107222868162096 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6809403624195333, + "performance_delta": 0.006323908913025034, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.1506961971471514, + "call_delta": -0.08215409919745909 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6695119530385957, + "performance_delta": -0.0017015001559708143, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.0, + "call_delta": -0.27691915038425546 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6797493123103536, + "performance_delta": 0.0057567421943680586, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.11924680259762357, + "call_delta": -0.15234098063125653 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7519496355953833, + "performance_delta": -0.006607492272353466, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.8643620792331906, + "call_delta": 0.29682670220911445 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7474406601820606, + "performance_delta": -0.010917959334146188, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9800073731673425, + "call_delta": 0.37016135893145785 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7482063352522474, + "performance_delta": -0.012194084451124354, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.96633865524771, + "call_delta": 0.36423446672149273 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9231541938724259, + "candidate_call": 1.0, + "call_delta": 0.07684580612757408 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9055750878955299, + "performance_delta": -0.0024395494008754293, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9716581760780656, + "call_delta": 0.03731075554279972 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.00021525435890079603, + "baseline_call": 0.8682643323527302, + "candidate_call": 1.0, + "call_delta": 0.13173566764726985 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9060055966133315, + "performance_delta": -0.001004520341536863, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9787615699217909, + "call_delta": 0.12850685226375824 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9060773480662984, + "performance_delta": 0.0022960464949415282, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9367152184831743, + "call_delta": 0.07246896749659182 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.000215254358900685, + "baseline_call": 0.98880677333716, + "candidate_call": 1.0, + "call_delta": 0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9033507928535552, + "performance_delta": 0.0033723182894452863, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9880892588074909, + "call_delta": 0.026404534691827575 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9023462725120184, + "performance_delta": 0.0017220348712061462, + "baseline_call": 1.0, + "candidate_call": 0.9800530960751955, + "call_delta": -0.01994690392480447 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9033507928535552, + "performance_delta": 0.0027265552127430093, + "baseline_call": 1.0, + "candidate_call": 0.987012987012987, + "call_delta": -0.012987012987012991 + } + ], + "rule": "endpoint" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_0.25", + "counts": { + "total": 48, + "performance_wins": 15, + "performance_ties": 14, + "performance_losses": 19, + "call_less": 5, + "call_ties": 14, + "call_more": 29, + "both_performance_win_and_call_less": 2, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.2211131170053352, + "performance_delta": -0.008624401383966807, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.988681757656458, + "call_delta": 0.19806924101198398 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.2222578228236749, + "performance_delta": -0.010196530694865202, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.9880159786950732, + "call_delta": 0.207057256990679 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.22347202518399215, + "performance_delta": -0.009296490065740876, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.988681757656458, + "call_delta": 0.20739014647137155 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.0016866851695898633, + "baseline_call": 0.8162450066577897, + "candidate_call": 1.0, + "call_delta": 0.18375499334221035 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.004083052908334106, + "baseline_call": 0.8392143808255659, + "candidate_call": 1.0, + "call_delta": 0.16078561917443412 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.00166953258981406, + "baseline_call": 0.829560585885486, + "candidate_call": 1.0, + "call_delta": 0.17043941411451402 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2587284729338324, + "performance_delta": -0.008605223956643926, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.5702396804260985, + "call_delta": 0.1984021304926764 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2591386172712909, + "performance_delta": 0.000526582405320497, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.5509320905459387, + "call_delta": 0.1914114513981358 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.25650251356280634, + "performance_delta": -0.00906698736235001, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5622503328894807, + "call_delta": 0.10785619174434086 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.2712680480298825, + "performance_delta": -0.02650163757066487, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.6844207723035952, + "call_delta": 0.1934087882822903 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.27348865863056027, + "performance_delta": -0.025900384275046706, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.6474700399467377, + "call_delta": 0.19307589880159787 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2704420670349152, + "performance_delta": -0.032529635155747916, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.5572569906790945, + "call_delta": 0.10252996005326226 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.043473328985055126, + "baseline_call": 0.407140628987891, + "candidate_call": 1.0, + "call_delta": 0.5928593710121091 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.03689419504863456, + "baseline_call": 0.4556050250971273, + "candidate_call": 1.0, + "call_delta": 0.5443949749028727 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.05373904659274592, + "baseline_call": 0.40413464537900917, + "candidate_call": 1.0, + "call_delta": 0.5958653546209909 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7447182599325072, + "performance_delta": -0.004622408757054108, + "baseline_call": 0.8589456370700167, + "candidate_call": 1.0, + "call_delta": 0.1410543629299833 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7447182599325072, + "performance_delta": 0.03210163627598339, + "baseline_call": 0.6885120381136035, + "candidate_call": 1.0, + "call_delta": 0.3114879618863965 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7447182599325072, + "performance_delta": 0.09026458327425346, + "baseline_call": 0.562374159884298, + "candidate_call": 1.0, + "call_delta": 0.43762584011570205 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6695119530385957, + "performance_delta": -0.005104500467912554, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.0, + "call_delta": -0.2328502963446105 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6695119530385957, + "performance_delta": -0.0017015001559708143, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.0, + "call_delta": -0.27691915038425546 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6695119530385957, + "performance_delta": -0.004480617077389892, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.0, + "call_delta": -0.2715877832288801 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7470436434790007, + "performance_delta": -0.011513484388736117, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.9068712247965289, + "call_delta": 0.33933584777245274 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7490287269943, + "performance_delta": -0.009329892521906746, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9408445112440802, + "call_delta": 0.3309984970081955 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7492272353458299, + "performance_delta": -0.011173184357541888, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.938831069392848, + "call_delta": 0.3367268808666307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9231541938724259, + "candidate_call": 1.0, + "call_delta": 0.07684580612757408 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9050728277247614, + "performance_delta": -0.0029418095716439163, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9550835904427064, + "call_delta": 0.020736169907440538 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9062926024251992, + "performance_delta": -0.0012915261534045541, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9650570424051087, + "call_delta": 0.09679271005237855 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9090191576379422, + "performance_delta": 0.0008610174356030731, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.9368587213891081, + "call_delta": 0.10956446868049075 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9069383655019014, + "performance_delta": -7.1751452966895e-05, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9668508287292817, + "call_delta": 0.11659611107124912 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9096649207146444, + "performance_delta": 0.005883619143287611, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8986151969577384, + "call_delta": 0.03436894597115592 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.000215254358900685, + "baseline_call": 0.98880677333716, + "candidate_call": 1.0, + "call_delta": 0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9027767812298199, + "performance_delta": 0.0027983066657099043, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9891655306019946, + "call_delta": 0.027480806486331333 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9030637870416875, + "performance_delta": 0.0024395494008753182, + "baseline_call": 1.0, + "candidate_call": 0.988017507354524, + "call_delta": -0.011982492645476017 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9031355384946546, + "performance_delta": 0.0025113008538423243, + "baseline_call": 1.0, + "candidate_call": 0.9648417880462079, + "call_delta": -0.035158211953792096 + } + ], + "rule": "endpoint" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_0.5", + "counts": { + "total": 48, + "performance_wins": 14, + "performance_ties": 14, + "performance_losses": 20, + "call_less": 5, + "call_ties": 14, + "call_more": 29, + "both_performance_win_and_call_less": 2, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22427095268610697, + "performance_delta": -0.0054665657031950365, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.9853528628495339, + "call_delta": 0.19474034620505987 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.22607173521804688, + "performance_delta": -0.00638261830049322, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.9833555259653795, + "call_delta": 0.20239680426098527 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.2255853947574774, + "performance_delta": -0.0071831204922556124, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.9810252996005326, + "call_delta": 0.19973368841544614 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.0016866851695898633, + "baseline_call": 0.8162450066577897, + "candidate_call": 1.0, + "call_delta": 0.18375499334221035 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.004083052908334106, + "baseline_call": 0.8392143808255659, + "candidate_call": 1.0, + "call_delta": 0.16078561917443412 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.00166953258981406, + "baseline_call": 0.829560585885486, + "candidate_call": 1.0, + "call_delta": 0.17043941411451402 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.26009300196494534, + "performance_delta": -0.007240694925530977, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.5699067909454061, + "call_delta": 0.19806924101198398 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2610246960878656, + "performance_delta": 0.002412661221895207, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.5223035952063915, + "call_delta": 0.16278295605858856 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2584417288296328, + "performance_delta": -0.007127772095523566, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5529294274300932, + "call_delta": 0.0985352862849534 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.2741835645844183, + "performance_delta": -0.023586121016129113, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.6231691078561917, + "call_delta": 0.1321571238348868 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2747498073529785, + "performance_delta": -0.02463923555262848, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.6195073235685752, + "call_delta": 0.16511318242343537 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2746117403663106, + "performance_delta": -0.02835996182435252, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.6088548601864181, + "call_delta": 0.15412782956058585 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.043473328985055126, + "baseline_call": 0.407140628987891, + "candidate_call": 1.0, + "call_delta": 0.5928593710121091 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.03689419504863456, + "baseline_call": 0.4556050250971273, + "candidate_call": 1.0, + "call_delta": 0.5443949749028727 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.05373904659274592, + "baseline_call": 0.40413464537900917, + "candidate_call": 1.0, + "call_delta": 0.5958653546209909 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7447182599325072, + "performance_delta": -0.004622408757054108, + "baseline_call": 0.8589456370700167, + "candidate_call": 1.0, + "call_delta": 0.1410543629299833 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7447182599325072, + "performance_delta": 0.03210163627598339, + "baseline_call": 0.6885120381136035, + "candidate_call": 1.0, + "call_delta": 0.3114879618863965 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7447182599325072, + "performance_delta": 0.09026458327425346, + "baseline_call": 0.562374159884298, + "candidate_call": 1.0, + "call_delta": 0.43762584011570205 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6695119530385957, + "performance_delta": -0.005104500467912554, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.019680685137396136, + "call_delta": -0.21316961120721437 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6695403113745285, + "performance_delta": -0.001673141820037971, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.019935910160791766, + "call_delta": -0.2569832402234637 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.669625386382327, + "performance_delta": -0.0043671837336585195, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.016929926551909932, + "call_delta": -0.25465785667697016 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7471287184867992, + "performance_delta": -0.011428409380937588, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.9701103139267788, + "call_delta": 0.40257493690270263 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7488018603068372, + "performance_delta": -0.009556759209369603, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.959107279584834, + "call_delta": 0.3492612653489493 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7490287269943, + "performance_delta": -0.01137169270907179, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9377818109633327, + "call_delta": 0.3356776224371154 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9066513596900337, + "performance_delta": -0.0014350290593384551, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.987012987012987, + "call_delta": 0.06385879314056109 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9075123771256368, + "performance_delta": -0.0005022601707684871, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9893807849608954, + "call_delta": 0.05503336442562956 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9069383655019014, + "performance_delta": -0.0006457630767022771, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9877305015426563, + "call_delta": 0.11946616918992614 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080863887493722, + "performance_delta": -7.1751452966895e-05, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.9687163665064218, + "call_delta": 0.14142211379780445 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9065796082370668, + "performance_delta": -0.00043050871780148103, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9649852909521418, + "call_delta": 0.11473057329410918 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9091626605438761, + "performance_delta": 0.0053813589725192346, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9574513883906149, + "call_delta": 0.09320513740403247 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.000215254358900685, + "baseline_call": 0.98880677333716, + "candidate_call": 1.0, + "call_delta": 0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9029202841357538, + "performance_delta": 0.0029418095716438053, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9893807849608954, + "call_delta": 0.02769606084523213 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9034942957594891, + "performance_delta": 0.0028700581186769103, + "baseline_call": 1.0, + "candidate_call": 0.9801965989811293, + "call_delta": -0.01980340101887068 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9028485326827869, + "performance_delta": 0.0022242950419746332, + "baseline_call": 1.0, + "candidate_call": 0.9671378345411494, + "call_delta": -0.03286216545885057 + } + ], + "rule": "endpoint" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_0.75", + "counts": { + "total": 48, + "performance_wins": 20, + "performance_ties": 11, + "performance_losses": 17, + "call_less": 11, + "call_ties": 11, + "call_more": 26, + "both_performance_win_and_call_less": 8, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.2319989174313225, + "performance_delta": 0.0022613990420204955, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7762982689747004, + "call_delta": -0.014314247669773628 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23674301754245702, + "performance_delta": 0.004288664023916922, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7596537949400799, + "call_delta": -0.02130492676431428 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.23714462398254227, + "performance_delta": 0.004376108732809247, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.8578561917443409, + "call_delta": 0.07656458055925441 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2698761268543865, + "performance_delta": 0.0015527213451139166, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8032623169107856, + "call_delta": -0.012982689747004006 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2684151526427204, + "performance_delta": 0.002488114872192082, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8169107856191744, + "call_delta": -0.02230359520639147 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.2696999750942859, + "performance_delta": 0.0013594170052375065, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8199067909454061, + "call_delta": -0.009653794940079896 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.26372894434686617, + "performance_delta": -0.0036047525436101457, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.3848202396804261, + "call_delta": 0.012982689747004006 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2621221451028754, + "performance_delta": 0.0035101102369050063, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.3725033288948069, + "call_delta": 0.012982689747004006 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.25716044964994544, + "performance_delta": -0.00840905127521091, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.40346205059920104, + "call_delta": -0.05093209054593878 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.29493861915351943, + "performance_delta": -0.0028310664470279634, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.5512649800266312, + "call_delta": 0.06025299600532624 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2927064243318904, + "performance_delta": -0.006682618573716548, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5825565912117177, + "call_delta": 0.12816245006657784 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2929922062010103, + "performance_delta": -0.009979495989652809, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.5808921438082557, + "call_delta": 0.12616511318242346 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8120469451156369, + "performance_delta": -0.0022437003797031174, + "baseline_call": 0.0, + "candidate_call": 0.06247842595788747, + "call_delta": 0.06247842595788747 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8198136002761477, + "performance_delta": 0.005522954780807665, + "baseline_call": 0.0, + "candidate_call": 0.07611322057300655, + "call_delta": 0.07611322057300655 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.8945460821539524, + "performance_delta": -0.007076285812909866, + "baseline_call": 0.0, + "candidate_call": 0.050914739385571284, + "call_delta": 0.050914739385571284 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48044692737430167, + "performance_delta": 0.023849360519524687, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5296202818818592, + "call_delta": 0.12247965289396817 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.4890111448260216, + "performance_delta": 0.025834444034824045, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.550860675495562, + "call_delta": 0.09525565039843464 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.4808723024132944, + "performance_delta": 0.034540453166208185, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5036156878314381, + "call_delta": 0.09948104245242889 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7464481184244108, + "performance_delta": -0.002892550265150451, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.9825879817372316, + "call_delta": 0.12364234466721491 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7467033434478065, + "performance_delta": 0.03408671979128275, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.9870402404786887, + "call_delta": 0.29852820236508515 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7468734934634036, + "performance_delta": 0.09241981680514988, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9880611405722712, + "call_delta": 0.4256869806879733 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.671865694921022, + "performance_delta": -0.002750758585486235, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.02064486855911295, + "call_delta": -0.21220542778549753 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6721492782803505, + "performance_delta": 0.0009358250857840478, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.018659785043813628, + "call_delta": -0.25825936534044186 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6705612114681111, + "performance_delta": -0.0034313586478744718, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.016306043161387288, + "call_delta": -0.2552817400674928 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7471287184867992, + "performance_delta": -0.011428409380937588, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.9700252389189802, + "call_delta": 0.402489861894904 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7486600686271729, + "performance_delta": -0.009698550889033819, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9587386212177069, + "call_delta": 0.34889260698182223 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7493690270254941, + "performance_delta": -0.011031392677877672, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9490117119927403, + "call_delta": 0.346907523466523 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9062926024251992, + "performance_delta": -0.0017937863241730412, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.987945755901557, + "call_delta": 0.06479156202913106 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9066513596900337, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9885915189782594, + "call_delta": 0.054244098442993494 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.00021525435890079603, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9881610102604578, + "call_delta": 0.11989667790772762 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9084451460142068, + "performance_delta": 0.00028700581186769103, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.9588146659969864, + "call_delta": 0.13152041328836905 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9062926024251992, + "performance_delta": -0.0007175145296691721, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.966563822917414, + "call_delta": 0.11630910525938143 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9085168974671737, + "performance_delta": 0.0047355958958168465, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9580254000143503, + "call_delta": 0.09377914902776785 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.000215254358900685, + "baseline_call": 0.98880677333716, + "candidate_call": 1.0, + "call_delta": 0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9031355384946546, + "performance_delta": 0.0031570639305446013, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9893090335079285, + "call_delta": 0.027624309392265234 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9039965559302576, + "performance_delta": 0.0033723182894453974, + "baseline_call": 1.0, + "candidate_call": 0.9791920786395925, + "call_delta": -0.020807921360407544 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.902705029776853, + "performance_delta": 0.002080792136040732, + "baseline_call": 1.0, + "candidate_call": 0.9555858506134749, + "call_delta": -0.044414149386525104 + } + ], + "rule": "endpoint" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_1", + "counts": { + "total": 48, + "performance_wins": 17, + "performance_ties": 10, + "performance_losses": 21, + "call_less": 14, + "call_ties": 11, + "call_more": 23, + "both_performance_win_and_call_less": 8, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22850906574549645, + "performance_delta": -0.0012284526438055554, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7906125166444741, + "call_delta": 0.0 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23004315338506715, + "performance_delta": -0.002411200133472957, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7926098535286284, + "call_delta": 0.011651131824234273 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.2308450652014655, + "performance_delta": -0.001923450048267522, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.7766311584553928, + "call_delta": -0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2678997047385957, + "performance_delta": -0.0004237007706768825, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8209054593874834, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2672954581839896, + "performance_delta": 0.0013684204134612465, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8215712383488681, + "call_delta": -0.017643142476697737 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.267918819975632, + "performance_delta": -0.0004217381134163922, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8149134487350199, + "call_delta": -0.01464713715046606 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2675187098549581, + "performance_delta": 0.00018501296448181215, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.4144474034620506, + "call_delta": 0.042609853528628505 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2578686516978949, + "performance_delta": -0.0007433831680754843, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.35918774966711053, + "call_delta": -0.00033288948069237767 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.26086905351963807, + "performance_delta": -0.004700447405518282, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.39980026631158455, + "call_delta": -0.05459387483355527 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.3029107097560361, + "performance_delta": 0.005141024155488727, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.4337549933422104, + "call_delta": -0.05725699067909451 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.29937410202312603, + "performance_delta": -1.4940882480940498e-05, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.44274300932090543, + "call_delta": -0.011651131824234384 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3009613147999525, + "performance_delta": -0.002010387390710633, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.4477363515312916, + "call_delta": -0.006990679094540653 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8125647221263376, + "performance_delta": -0.0017259233690024578, + "baseline_call": 0.0, + "candidate_call": 0.017604418363824648, + "call_delta": 0.017604418363824648 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8134276838108389, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.0, + "candidate_call": 0.010700724887814981, + "call_delta": 0.010700724887814981 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.8998964445978599, + "performance_delta": -0.0017259233690023468, + "baseline_call": 0.0, + "candidate_call": 0.027614773904038662, + "call_delta": 0.027614773904038662 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.901449775629962, + "performance_delta": -0.00017259233690025688, + "baseline_call": 0.0, + "candidate_call": 0.010873317224715223, + "call_delta": 0.010873317224715223 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48586336953747555, + "performance_delta": 0.029265802682698572, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5045798712531548, + "call_delta": 0.09743924226526385 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.49794402064486853, + "performance_delta": 0.034767319853670986, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.5430054164421632, + "call_delta": 0.08740039134503585 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.49649774551229336, + "performance_delta": 0.05016589626520718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5379576326461163, + "call_delta": 0.13382298726710712 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7511272438533307, + "performance_delta": 0.001786575163769455, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.852763519836656, + "call_delta": -0.006182117233360707 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7536511357513541, + "performance_delta": 0.04103451209483033, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8933726568924936, + "call_delta": 0.20486061877889006 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7511839605251964, + "performance_delta": 0.09673028386694271, + "baseline_call": 0.562374159884298, + "candidate_call": 0.8652411876471089, + "call_delta": 0.30286702776281094 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6800045373337492, + "performance_delta": 0.005388083827240986, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.12287666959702805, + "call_delta": -0.10997362674758245 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6805717040524062, + "performance_delta": 0.0093582508578397, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.09480191702350906, + "call_delta": -0.18211723336074642 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6810254374273318, + "performance_delta": 0.007032867311346225, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.12109009443325866, + "call_delta": -0.15049768879562145 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7519496355953833, + "performance_delta": -0.006607492272353466, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.8643620792331906, + "call_delta": 0.29682670220911445 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7473555851742619, + "performance_delta": -0.011003034341944828, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9705640473017043, + "call_delta": 0.3607180330658196 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7550406942120637, + "performance_delta": -0.005359725491308143, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9094518333664181, + "call_delta": 0.30734764484020083 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9060773480662984, + "performance_delta": -0.002009040683073837, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9890937791490277, + "call_delta": 0.06593958527660182 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0015785319652723562, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9886632704312263, + "call_delta": 0.05431584989596039 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9071536198608022, + "performance_delta": -0.00043050871780148103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.98973954222573, + "call_delta": 0.12147520987299987 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9052163306306953, + "performance_delta": -0.0017937863241730412, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9674248403530171, + "call_delta": 0.1171701226949845 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9075123771256368, + "performance_delta": 0.0037310755542799834, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9586711630910526, + "call_delta": 0.09442491210447013 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.000215254358900685, + "baseline_call": 0.98880677333716, + "candidate_call": 1.0, + "call_delta": 0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9034225443065222, + "performance_delta": 0.0034440697424122924, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9885197675252924, + "call_delta": 0.026835043409629056 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9037095501183899, + "performance_delta": 0.0030853124775777063, + "baseline_call": 1.0, + "candidate_call": 0.9783310612039894, + "call_delta": -0.021668938796010617 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9026332783238861, + "performance_delta": 0.002009040683073837, + "baseline_call": 1.0, + "candidate_call": 0.9682141063356533, + "call_delta": -0.0317858936643467 + } + ], + "rule": "endpoint" + }, + { + "rule": "endpoint", + "baseline": "original", + "candidate": "confidence", + "counts": { + "total": 48, + "performance_wins": 12, + "performance_ties": 12, + "performance_losses": 24, + "call_less": 12, + "call_ties": 12, + "call_more": 24, + "both_performance_win_and_call_less": 3, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.05373904659274592, + "baseline_call": 0.40413464537900917, + "candidate_call": 1.0, + "call_delta": 0.5958653546209909 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.05373904659274592, + "baseline_call": 0.40413464537900917, + "candidate_call": 1.0, + "call_delta": 0.5958653546209909 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.05373904659274592, + "baseline_call": 0.40413464537900917, + "candidate_call": 1.0, + "call_delta": 0.5958653546209909 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7447182599325072, + "performance_delta": 0.09026458327425346, + "baseline_call": 0.562374159884298, + "candidate_call": 1.0, + "call_delta": 0.43762584011570205 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7447182599325072, + "performance_delta": 0.09026458327425346, + "baseline_call": 0.562374159884298, + "candidate_call": 1.0, + "call_delta": 0.43762584011570205 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7447182599325072, + "performance_delta": 0.09026458327425346, + "baseline_call": 0.562374159884298, + "candidate_call": 1.0, + "call_delta": 0.43762584011570205 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6187788900547316, + "performance_delta": -0.055213680061253934, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.5209426310864078, + "call_delta": 0.2493548478575277 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6187788900547316, + "performance_delta": -0.055213680061253934, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.5209426310864078, + "call_delta": 0.2493548478575277 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6187788900547316, + "performance_delta": -0.055213680061253934, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.5209426310864078, + "call_delta": 0.2493548478575277 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7243286163967898, + "performance_delta": -0.036071803306581995, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6838612710206166, + "call_delta": 0.08175708249439928 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7243286163967898, + "performance_delta": -0.036071803306581995, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6838612710206166, + "call_delta": 0.08175708249439928 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7243286163967898, + "performance_delta": -0.036071803306581995, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6838612710206166, + "call_delta": 0.08175708249439928 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9077276314845376, + "performance_delta": 0.00014350290593390103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8580038745784602, + "call_delta": -0.010260457774269982 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9077276314845376, + "performance_delta": 0.00014350290593390103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8580038745784602, + "call_delta": -0.010260457774269982 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9077276314845376, + "performance_delta": 0.00014350290593390103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8580038745784602, + "call_delta": -0.010260457774269982 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9024180239649853, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8580038745784602, + "call_delta": -0.006242376408122308 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9024180239649853, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8580038745784602, + "call_delta": -0.006242376408122308 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9024180239649853, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8580038745784602, + "call_delta": -0.006242376408122308 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.22361115727328065, + "performance_delta": -0.009157357976452368, + "baseline_call": 0.7812916111850865, + "candidate_call": 1.0, + "call_delta": 0.2187083888149135 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.22361115727328065, + "performance_delta": -0.009157357976452368, + "baseline_call": 0.7812916111850865, + "candidate_call": 1.0, + "call_delta": 0.2187083888149135 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.22361115727328065, + "performance_delta": -0.009157357976452368, + "baseline_call": 0.7812916111850865, + "candidate_call": 1.0, + "call_delta": 0.2187083888149135 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.00166953258981406, + "baseline_call": 0.829560585885486, + "candidate_call": 1.0, + "call_delta": 0.17043941411451402 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.00166953258981406, + "baseline_call": 0.829560585885486, + "candidate_call": 1.0, + "call_delta": 0.17043941411451402 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.00166953258981406, + "baseline_call": 0.829560585885486, + "candidate_call": 1.0, + "call_delta": 0.17043941411451402 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0009327688885699681, + "baseline_call": 1.0, + "candidate_call": 0.9593169261677549, + "call_delta": -0.04068307383224512 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0009327688885699681, + "baseline_call": 1.0, + "candidate_call": 0.9593169261677549, + "call_delta": -0.04068307383224512 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0009327688885699681, + "baseline_call": 1.0, + "candidate_call": 0.9593169261677549, + "call_delta": -0.04068307383224512 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.8996197172992753, + "performance_delta": -0.0010045203415369741, + "baseline_call": 1.0, + "candidate_call": 0.9593169261677549, + "call_delta": -0.04068307383224512 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.8996197172992753, + "performance_delta": -0.0010045203415369741, + "baseline_call": 1.0, + "candidate_call": 0.9593169261677549, + "call_delta": -0.04068307383224512 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.8996197172992753, + "performance_delta": -0.0010045203415369741, + "baseline_call": 1.0, + "candidate_call": 0.9593169261677549, + "call_delta": -0.04068307383224512 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2523047077304935, + "performance_delta": -0.013264793194662838, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.9254327563249002, + "call_delta": 0.47103861517976037 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2523047077304935, + "performance_delta": -0.013264793194662838, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.9254327563249002, + "call_delta": 0.47103861517976037 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2523047077304935, + "performance_delta": -0.013264793194662838, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.9254327563249002, + "call_delta": 0.47103861517976037 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2597490004762105, + "performance_delta": -0.04322270171445258, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.9221038615179761, + "call_delta": 0.4673768308921438 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2597490004762105, + "performance_delta": -0.04322270171445258, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.9221038615179761, + "call_delta": 0.4673768308921438 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2597490004762105, + "performance_delta": -0.04322270171445258, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.9221038615179761, + "call_delta": 0.4673768308921438 + } + ] + } +] \ No newline at end of file diff --git a/paper_results/data/earlier_results.json b/paper_results/data/earlier_results.json new file mode 100644 index 0000000000000000000000000000000000000000..28fc0dede625517749a11d9342fa6c5f9b30cfd7 --- /dev/null +++ b/paper_results/data/earlier_results.json @@ -0,0 +1,6448 @@ +{ + "status": "complete", + "planned": 264, + "completed": 264, + "missing": [], + "rows": [ + { + "id": "formal_0205", + "tag": "baseline_25pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12161748372078302, + "fixed25_actual_fraction": 0.26897470039946736, + "fixed25_rescue": 1103, + "fixed25_harm": 518, + "rank25_metric": 0.12103479129839005, + "validation_selection": 0.13942018347651794, + "best_epoch": 1, + "seconds": 3.9248127937316895 + }, + { + "id": "formal_0206", + "tag": "baseline_50pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.11138731664705415, + "fixed25_actual_fraction": 0.25798934753661784, + "fixed25_rescue": 1337, + "fixed25_harm": 464, + "rank25_metric": 0.11157434949576527, + "validation_selection": 0.10286119045745137, + "best_epoch": 3, + "seconds": 5.573187828063965 + }, + { + "id": "extra_0031", + "tag": "representation_32", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12677122948456399, + "fixed25_actual_fraction": 0.25099866844207724, + "fixed25_rescue": 1352, + "fixed25_harm": 445, + "rank25_metric": 0.12664773557636813, + "validation_selection": 0.10657501019916582, + "best_epoch": 16, + "seconds": 17.047492742538452 + }, + { + "id": "extra_0032", + "tag": "representation_64", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.1199038046999191, + "fixed25_actual_fraction": 0.24500665778961384, + "fixed25_rescue": 1268, + "fixed25_harm": 454, + "rank25_metric": 0.12123578062564436, + "validation_selection": 0.10492587324457393, + "best_epoch": 1, + "seconds": 6.62996768951416 + }, + { + "id": "extra_0033", + "tag": "student_8M", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12852176195071466, + "fixed25_actual_fraction": 0.28129161118508655, + "fixed25_rescue": 1430, + "fixed25_harm": 528, + "rank25_metric": 0.12791702188360424, + "validation_selection": 0.1101086951271347, + "best_epoch": 16, + "seconds": 17.678598165512085 + }, + { + "id": "formal_0114", + "tag": "baseline_100pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.11409050364241882, + "fixed25_actual_fraction": 0.2333555259653795, + "fixed25_rescue": 1155, + "fixed25_harm": 428, + "rank25_metric": 0.1142018021781044, + "validation_selection": 0.09196748903785351, + "best_epoch": 1, + "seconds": 7.1226747035980225 + }, + { + "id": "formal_0115", + "tag": "drop_image", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12791159721178413, + "fixed25_actual_fraction": 0.2626498002663116, + "fixed25_rescue": 1369, + "fixed25_harm": 464, + "rank25_metric": 0.12685189507799977, + "validation_selection": 0.12058376950960199, + "best_epoch": 47, + "seconds": 33.72947406768799 + }, + { + "id": "formal_0116", + "tag": "drop_text", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.11259143129519858, + "fixed25_actual_fraction": 0.25, + "fixed25_rescue": 1237, + "fixed25_harm": 472, + "rank25_metric": 0.11259143129519858, + "validation_selection": 0.10775056216618559, + "best_epoch": 1, + "seconds": 7.603894233703613 + }, + { + "id": "formal_0117", + "tag": "drop_hidden", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12396779300064459, + "fixed25_actual_fraction": 0.2589880159786951, + "fixed25_rescue": 1318, + "fixed25_harm": 473, + "rank25_metric": 0.12250988658190227, + "validation_selection": 0.09268882908524566, + "best_epoch": 4, + "seconds": 9.375170469284058 + }, + { + "id": "formal_0118", + "tag": "drop_confidence", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.1079733386177777, + "fixed25_actual_fraction": 0.24866844207723035, + "fixed25_rescue": 1260, + "fixed25_harm": 450, + "rank25_metric": 0.1079857737928408, + "validation_selection": 0.0975193305968221, + "best_epoch": 1, + "seconds": 7.336633205413818 + }, + { + "id": "formal_0119", + "tag": "hidden_position_feature_blocks__layer_03", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_03", + "fixed25_metric": 0.12865126241054672, + "fixed25_actual_fraction": 0.25, + "fixed25_rescue": 1285, + "fixed25_harm": 410, + "rank25_metric": 0.12865126241054672, + "validation_selection": 0.11612297444582934, + "best_epoch": 15, + "seconds": 17.645463705062866 + }, + { + "id": "formal_0120", + "tag": "hidden_position_feature_blocks__layer_24", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_24", + "fixed25_metric": 0.13130687829759294, + "fixed25_actual_fraction": 0.28395472703062585, + "fixed25_rescue": 1454, + "fixed25_harm": 484, + "rank25_metric": 0.12813467068447604, + "validation_selection": 0.11472043231101983, + "best_epoch": 5, + "seconds": 10.408715963363647 + }, + { + "id": "formal_0121", + "tag": "hidden_multi_layer", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_03", + "fixed25_metric": 0.1319786802729061, + "fixed25_actual_fraction": 0.28961384820239683, + "fixed25_rescue": 1468, + "fixed25_harm": 495, + "rank25_metric": 0.1291292294447022, + "validation_selection": 0.12972548686634056, + "best_epoch": 26, + "seconds": 25.749829053878784 + }, + { + "id": "formal_0122", + "tag": "representation_128", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12735166754382224, + "fixed25_actual_fraction": 0.2539946737683089, + "fixed25_rescue": 1334, + "fixed25_harm": 491, + "rank25_metric": 0.12825332409803333, + "validation_selection": 0.10683973009120012, + "best_epoch": 1, + "seconds": 7.299476146697998 + }, + { + "id": "formal_0123", + "tag": "representation_256", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.1296190849296716, + "fixed25_actual_fraction": 0.25798934753661784, + "fixed25_rescue": 1349, + "fixed25_harm": 478, + "rank25_metric": 0.12942150404041414, + "validation_selection": 0.1047735090992197, + "best_epoch": 4, + "seconds": 9.053397178649902 + }, + { + "id": "formal_0124", + "tag": "student_1M", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12038804662706637, + "fixed25_actual_fraction": 0.24167776298268975, + "fixed25_rescue": 1274, + "fixed25_harm": 442, + "rank25_metric": 0.12037820583822222, + "validation_selection": 0.1010803735167908, + "best_epoch": 5, + "seconds": 11.015799045562744 + }, + { + "id": "formal_0125", + "tag": "student_2M", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12899655423992204, + "fixed25_actual_fraction": 0.27529960053262315, + "fixed25_rescue": 1369, + "fixed25_harm": 532, + "rank25_metric": 0.12740185163408788, + "validation_selection": 0.11096710446690258, + "best_epoch": 16, + "seconds": 17.74562120437622 + }, + { + "id": "formal_0126", + "tag": "target_error", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12363065005052852, + "fixed25_actual_fraction": 0.26531291611185087, + "fixed25_rescue": 1430, + "fixed25_harm": 491, + "rank25_metric": 0.12410125357669675, + "validation_selection": 0.1136359842515143, + "best_epoch": 36, + "seconds": 33.95516061782837 + }, + { + "id": "formal_0127", + "tag": "target_four_state", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12288572047662566, + "fixed25_actual_fraction": 0.26198402130492676, + "fixed25_rescue": 1220, + "fixed25_harm": 355, + "rank25_metric": 0.12364678175576603, + "validation_selection": 0.13440378191215316, + "best_epoch": 15, + "seconds": 17.694798231124878 + }, + { + "id": "formal_0128", + "tag": "fusion_attention", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.1205106128632194, + "fixed25_actual_fraction": 0.25798934753661784, + "fixed25_rescue": 1335, + "fixed25_harm": 490, + "rank25_metric": 0.11993843825140044, + "validation_selection": 0.11337000715277953, + "best_epoch": 16, + "seconds": 24.939371585845947 + }, + { + "id": "formal_0137", + "tag": "target_error", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.13298608758280392, + "fixed25_actual_fraction": 0.27729693741677763, + "fixed25_rescue": 1315, + "fixed25_harm": 506, + "rank25_metric": 0.13302267602819143, + "validation_selection": 0.12208124766901975, + "best_epoch": 13, + "seconds": 13.368404150009155 + }, + { + "id": "formal_0207", + "tag": "baseline_25pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.14471609214350373, + "fixed25_actual_fraction": 0.2892809587217044, + "fixed25_rescue": 1383, + "fixed25_harm": 258, + "rank25_metric": 0.14346432207238047, + "validation_selection": 0.16233096000534752, + "best_epoch": 17, + "seconds": 8.028136253356934 + }, + { + "id": "formal_0208", + "tag": "baseline_50pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1499641421353766, + "fixed25_actual_fraction": 0.26797603195739017, + "fixed25_rescue": 1348, + "fixed25_harm": 225, + "rank25_metric": 0.14991013725904712, + "validation_selection": 0.14960536227871302, + "best_epoch": 14, + "seconds": 7.761091709136963 + }, + { + "id": "extra_0034", + "tag": "representation_32", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 32, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.15246154351186944, + "fixed25_actual_fraction": 0.26531291611185087, + "fixed25_rescue": 1381, + "fixed25_harm": 233, + "rank25_metric": 0.15246107875534035, + "validation_selection": 0.13959259383410275, + "best_epoch": 11, + "seconds": 9.594881772994995 + }, + { + "id": "extra_0035", + "tag": "representation_64", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 64, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.14746508696753746, + "fixed25_actual_fraction": 0.27463382157123833, + "fixed25_rescue": 1305, + "fixed25_harm": 253, + "rank25_metric": 0.14258824887550908, + "validation_selection": 0.13878028268730078, + "best_epoch": 1, + "seconds": 6.531559228897095 + }, + { + "id": "extra_0036", + "tag": "student_8M", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1498069310931111, + "fixed25_actual_fraction": 0.2539946737683089, + "fixed25_rescue": 1284, + "fixed25_harm": 203, + "rank25_metric": 0.15030144907600507, + "validation_selection": 0.14151521056562297, + "best_epoch": 16, + "seconds": 10.990516424179077 + }, + { + "id": "formal_0126", + "tag": "target_error", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.13996613214172965, + "fixed25_actual_fraction": 0.26564580559254325, + "fixed25_rescue": 1525, + "fixed25_harm": 248, + "rank25_metric": 0.13802590609803905, + "validation_selection": 0.13929873372091645, + "best_epoch": 30, + "seconds": 33.95516061782837 + }, + { + "id": "formal_0129", + "tag": "baseline_100pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1498069310931111, + "fixed25_actual_fraction": 0.2539946737683089, + "fixed25_rescue": 1284, + "fixed25_harm": 203, + "rank25_metric": 0.15030144907600507, + "validation_selection": 0.14151521056562297, + "best_epoch": 16, + "seconds": 13.48578929901123 + }, + { + "id": "formal_0130", + "tag": "hidden_position_feature_blocks__layer_03", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_03", + "fixed25_metric": 0.15010748535267285, + "fixed25_actual_fraction": 0.2513315579227696, + "fixed25_rescue": 1192, + "fixed25_harm": 205, + "rank25_metric": 0.14998631841175578, + "validation_selection": 0.14027716877702798, + "best_epoch": 12, + "seconds": 11.686585903167725 + }, + { + "id": "formal_0131", + "tag": "hidden_position_feature_blocks__layer_24", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_24", + "fixed25_metric": 0.13107435550044824, + "fixed25_actual_fraction": 0.2643142476697736, + "fixed25_rescue": 1339, + "fixed25_harm": 193, + "rank25_metric": 0.12841698578626595, + "validation_selection": 0.1314715107927409, + "best_epoch": 1, + "seconds": 6.488433599472046 + }, + { + "id": "formal_0132", + "tag": "hidden_multi_layer", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_03", + "fixed25_metric": 0.1465080909816311, + "fixed25_actual_fraction": 0.27463382157123833, + "fixed25_rescue": 1540, + "fixed25_harm": 254, + "rank25_metric": 0.14337516260382116, + "validation_selection": 0.1327068555595483, + "best_epoch": 9, + "seconds": 10.790199518203735 + }, + { + "id": "formal_0133", + "tag": "representation_128", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 128, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.14803489232686926, + "fixed25_actual_fraction": 0.2739680426098535, + "fixed25_rescue": 1403, + "fixed25_harm": 224, + "rank25_metric": 0.14798656495937493, + "validation_selection": 0.14100279346596564, + "best_epoch": 10, + "seconds": 10.514626741409302 + }, + { + "id": "formal_0134", + "tag": "representation_256", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 256, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.14730603556730637, + "fixed25_actual_fraction": 0.2626498002663116, + "fixed25_rescue": 1281, + "fixed25_harm": 261, + "rank25_metric": 0.1447121436747074, + "validation_selection": 0.14000847459984167, + "best_epoch": 1, + "seconds": 5.760014295578003 + }, + { + "id": "formal_0135", + "tag": "student_1M", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1498069310931111, + "fixed25_actual_fraction": 0.2539946737683089, + "fixed25_rescue": 1284, + "fixed25_harm": 203, + "rank25_metric": 0.15030144907600507, + "validation_selection": 0.14151521056562297, + "best_epoch": 16, + "seconds": 12.58092188835144 + }, + { + "id": "formal_0136", + "tag": "student_2M", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1498069310931111, + "fixed25_actual_fraction": 0.2539946737683089, + "fixed25_rescue": 1284, + "fixed25_harm": 203, + "rank25_metric": 0.15030144907600507, + "validation_selection": 0.14151521056562297, + "best_epoch": 16, + "seconds": 13.341734886169434 + }, + { + "id": "formal_0137", + "tag": "target_error", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1550677773614836, + "fixed25_actual_fraction": 0.2872836218375499, + "fixed25_rescue": 1438, + "fixed25_harm": 248, + "rank25_metric": 0.15400963778422433, + "validation_selection": 0.14514986541925773, + "best_epoch": 10, + "seconds": 13.368404150009155 + }, + { + "id": "formal_0138", + "tag": "target_four_state", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1008636659774502, + "fixed25_actual_fraction": 0.26131824234354195, + "fixed25_rescue": 873, + "fixed25_harm": 87, + "rank25_metric": 0.10092160765502416, + "validation_selection": 0.11724983142173177, + "best_epoch": 1, + "seconds": 5.6207404136657715 + }, + { + "id": "formal_0213", + "tag": "baseline_25pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22250151381741792, + "validation_selection": 0.245059255252362, + "best_epoch": 2, + "seconds": 2.557858467102051 + }, + { + "id": "formal_0214", + "tag": "baseline_50pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22250151381741792, + "validation_selection": 0.2411747600492345, + "best_epoch": 1, + "seconds": 3.123697280883789 + }, + { + "id": "extra_0043", + "tag": "representation_32", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.24346726209631106, + "fixed25_actual_fraction": 0.4374167776298269, + "fixed25_rescue": 1092, + "fixed25_harm": 880, + "rank25_metric": 0.2235506221639314, + "validation_selection": 0.2322722938123662, + "best_epoch": 8, + "seconds": 7.180579900741577 + }, + { + "id": "extra_0044", + "tag": "representation_64", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.2235506221639314, + "validation_selection": 0.23309404497704123, + "best_epoch": 4, + "seconds": 6.816771030426025 + }, + { + "id": "extra_0045", + "tag": "student_8M", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.22272342953859628, + "validation_selection": 0.23173775721145673, + "best_epoch": 4, + "seconds": 7.2654266357421875 + }, + { + "id": "formal_0169", + "tag": "baseline_100pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.22272342953859628, + "validation_selection": 0.23173775721145673, + "best_epoch": 4, + "seconds": 9.216264009475708 + }, + { + "id": "formal_0170", + "tag": "representation_128", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.2235506221639314, + "validation_selection": 0.23309404497704123, + "best_epoch": 4, + "seconds": 8.15360426902771 + }, + { + "id": "formal_0171", + "tag": "representation_256", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.2235506221639314, + "validation_selection": 0.23309404497704123, + "best_epoch": 4, + "seconds": 9.3523588180542 + }, + { + "id": "formal_0172", + "tag": "student_1M", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.22272342953859628, + "validation_selection": 0.23173775721145673, + "best_epoch": 4, + "seconds": 8.9949951171875 + }, + { + "id": "formal_0173", + "tag": "student_2M", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.22272342953859628, + "validation_selection": 0.23173775721145673, + "best_epoch": 4, + "seconds": 9.508620262145996 + }, + { + "id": "formal_0174", + "tag": "target_error", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22250151381741792, + "validation_selection": 0.23040572580934265, + "best_epoch": 4, + "seconds": 9.888505697250366 + }, + { + "id": "formal_0175", + "tag": "target_four_state", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22322830961852883, + "validation_selection": 0.22881953100143768, + "best_epoch": 3, + "seconds": 6.8639256954193115 + }, + { + "id": "formal_0176", + "tag": "harm_cost_2", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22250151381741792, + "validation_selection": 0.23040572580934265, + "best_epoch": 7, + "seconds": 10.785712957382202 + }, + { + "id": "formal_0182", + "tag": "target_error", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22250151381741792, + "validation_selection": 0.23040572580934265, + "best_epoch": 4, + "seconds": 9.399006128311157 + }, + { + "id": "formal_0215", + "tag": "baseline_25pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.2490976644605033, + "best_epoch": 7, + "seconds": 3.354280948638916 + }, + { + "id": "formal_0216", + "tag": "baseline_50pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.2490976644605033, + "best_epoch": 14, + "seconds": 4.801910877227783 + }, + { + "id": "extra_0046", + "tag": "representation_32", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2917660075719453, + "fixed25_actual_fraction": 0.43209054593874835, + "fixed25_rescue": 1103, + "fixed25_harm": 315, + "rank25_metric": 0.2631043176189476, + "validation_selection": 0.24807135884361062, + "best_epoch": 8, + "seconds": 9.370010614395142 + }, + { + "id": "extra_0047", + "tag": "representation_64", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.2490976644605033, + "best_epoch": 3, + "seconds": 5.145432233810425 + }, + { + "id": "extra_0048", + "tag": "student_8M", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.24929743985848513, + "best_epoch": 13, + "seconds": 7.741400241851807 + }, + { + "id": "formal_0174", + "tag": "target_error", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.249028639438933, + "best_epoch": 4, + "seconds": 9.888505697250366 + }, + { + "id": "formal_0177", + "tag": "baseline_100pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.24929743985848513, + "best_epoch": 13, + "seconds": 11.173577785491943 + }, + { + "id": "formal_0178", + "tag": "representation_128", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.24929743985848513, + "best_epoch": 13, + "seconds": 13.545645475387573 + }, + { + "id": "formal_0179", + "tag": "representation_256", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.24929743985848513, + "best_epoch": 13, + "seconds": 12.355992555618286 + }, + { + "id": "formal_0180", + "tag": "student_1M", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.24929743985848513, + "best_epoch": 13, + "seconds": 15.147192239761353 + }, + { + "id": "formal_0181", + "tag": "student_2M", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.24929743985848513, + "best_epoch": 13, + "seconds": 11.001330137252808 + }, + { + "id": "formal_0182", + "tag": "target_error", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.249028639438933, + "best_epoch": 4, + "seconds": 9.399006128311157 + }, + { + "id": "formal_0183", + "tag": "target_four_state", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.24575590452405222, + "fixed25_actual_fraction": 0.21837549933422104, + "fixed25_rescue": 716, + "fixed25_harm": 113, + "rank25_metric": 0.26408669760452147, + "validation_selection": 0.23738117267347428, + "best_epoch": 4, + "seconds": 8.231167554855347 + }, + { + "id": "formal_0184", + "tag": "harm_cost_2", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.24575590452405222, + "fixed25_actual_fraction": 0.21837549933422104, + "fixed25_rescue": 716, + "fixed25_harm": 113, + "rank25_metric": 0.26408669760452147, + "validation_selection": 0.23758094807145608, + "best_epoch": 13, + "seconds": 10.565741539001465 + }, + { + "id": "formal_0185", + "tag": "baseline_25pct", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7849499482222989, + "fixed25_actual_fraction": 0.2587159130134622, + "fixed25_rescue": 31, + "fixed25_harm": 201, + "rank25_metric": 0.7884017949603037, + "validation_selection": 0.9741666666666666, + "best_epoch": 12, + "seconds": 5.802937984466553 + }, + { + "id": "formal_0186", + "tag": "baseline_50pct", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7904729030031067, + "fixed25_actual_fraction": 0.2549188816016569, + "fixed25_rescue": 38, + "fixed25_harm": 176, + "rank25_metric": 0.7930617880566103, + "validation_selection": 0.9731666666666667, + "best_epoch": 7, + "seconds": 7.1614015102386475 + }, + { + "id": "extra_0001", + "tag": "representation_32", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 32, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.787193648602002, + "fixed25_actual_fraction": 0.2557818432861581, + "fixed25_rescue": 33, + "fixed25_harm": 190, + "rank25_metric": 0.7916810493614084, + "validation_selection": 0.9744166666666666, + "best_epoch": 7, + "seconds": 5.997061252593994 + }, + { + "id": "extra_0002", + "tag": "representation_64", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 64, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7846047635484984, + "fixed25_actual_fraction": 0.2613047980669658, + "fixed25_rescue": 34, + "fixed25_harm": 206, + "rank25_metric": 0.7889195719710045, + "validation_selection": 0.9743333333333333, + "best_epoch": 12, + "seconds": 8.659805059432983 + }, + { + "id": "extra_0003", + "tag": "student_8M", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.787193648602002, + "fixed25_actual_fraction": 0.2557818432861581, + "fixed25_rescue": 33, + "fixed25_harm": 190, + "rank25_metric": 0.7913358646876079, + "validation_selection": 0.9744166666666666, + "best_epoch": 7, + "seconds": 7.315621376037598 + }, + { + "id": "formal_0001", + "tag": "baseline_100pct", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.787193648602002, + "fixed25_actual_fraction": 0.2557818432861581, + "fixed25_rescue": 33, + "fixed25_harm": 190, + "rank25_metric": 0.7913358646876079, + "validation_selection": 0.9744166666666666, + "best_epoch": 7, + "seconds": 2.9200518131256104 + }, + { + "id": "formal_0002", + "tag": "representation_128", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 128, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7846047635484984, + "fixed25_actual_fraction": 0.2613047980669658, + "fixed25_rescue": 34, + "fixed25_harm": 206, + "rank25_metric": 0.7913358646876079, + "validation_selection": 0.9744999999999999, + "best_epoch": 9, + "seconds": 3.735602378845215 + }, + { + "id": "formal_0003", + "tag": "representation_256", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 256, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7877114256127028, + "fixed25_actual_fraction": 0.2542285122540559, + "fixed25_rescue": 30, + "fixed25_harm": 184, + "rank25_metric": 0.7908180876769072, + "validation_selection": 0.9745833333333332, + "best_epoch": 16, + "seconds": 4.4355103969573975 + }, + { + "id": "formal_0004", + "tag": "student_1M", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.787193648602002, + "fixed25_actual_fraction": 0.2557818432861581, + "fixed25_rescue": 33, + "fixed25_harm": 190, + "rank25_metric": 0.7913358646876079, + "validation_selection": 0.9744166666666666, + "best_epoch": 7, + "seconds": 2.9487767219543457 + }, + { + "id": "formal_0005", + "tag": "student_2M", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.787193648602002, + "fixed25_actual_fraction": 0.2557818432861581, + "fixed25_rescue": 33, + "fixed25_harm": 190, + "rank25_metric": 0.7913358646876079, + "validation_selection": 0.9744166666666666, + "best_epoch": 7, + "seconds": 3.1687443256378174 + }, + { + "id": "formal_0006", + "tag": "target_error", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6755264066275457, + "fixed25_actual_fraction": 0.25716258198136005, + "fixed25_rescue": 89, + "fixed25_harm": 893, + "rank25_metric": 0.6810493614083535, + "validation_selection": 0.8948333333333334, + "best_epoch": 8, + "seconds": 3.2323741912841797 + }, + { + "id": "formal_0007", + "tag": "target_four_state", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7863306869175009, + "fixed25_actual_fraction": 0.2569899896444598, + "fixed25_rescue": 35, + "fixed25_harm": 197, + "rank25_metric": 0.7906454953400069, + "validation_selection": 0.9722500000000001, + "best_epoch": 2, + "seconds": 2.0133934020996094 + }, + { + "id": "formal_0008", + "tag": "harm_cost_2", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.787193648602002, + "fixed25_actual_fraction": 0.2557818432861581, + "fixed25_rescue": 33, + "fixed25_harm": 190, + "rank25_metric": 0.7913358646876079, + "validation_selection": 0.9745833333333332, + "best_epoch": 7, + "seconds": 3.1982953548431396 + }, + { + "id": "formal_0016", + "tag": "target_error", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.5395236451501554, + "fixed25_actual_fraction": 0.5980324473593372, + "fixed25_rescue": 227, + "fixed25_harm": 1819, + "rank25_metric": 0.7454263030721436, + "validation_selection": 0.9059999999999999, + "best_epoch": 2, + "seconds": 2.198946475982666 + }, + { + "id": "formal_0187", + "tag": "baseline_25pct", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6874352778736624, + "fixed25_actual_fraction": 0.6039005868139454, + "fixed25_rescue": 358, + "fixed25_harm": 1093, + "rank25_metric": 0.8060062133241284, + "validation_selection": 0.9316666666666666, + "best_epoch": 5, + "seconds": 4.623678922653198 + }, + { + "id": "formal_0188", + "tag": "baseline_50pct", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.694338971349672, + "fixed25_actual_fraction": 0.5997583707283397, + "fixed25_rescue": 357, + "fixed25_harm": 1052, + "rank25_metric": 0.8096306523990335, + "validation_selection": 0.9334166666666667, + "best_epoch": 3, + "seconds": 4.7386627197265625 + }, + { + "id": "extra_0004", + "tag": "representation_32", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 32, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7309285467725233, + "fixed25_actual_fraction": 0.5091473938557128, + "fixed25_rescue": 334, + "fixed25_harm": 817, + "rank25_metric": 0.7766655160510874, + "validation_selection": 0.9439166666666667, + "best_epoch": 16, + "seconds": 9.204903364181519 + }, + { + "id": "extra_0005", + "tag": "representation_64", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 64, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7074559889540905, + "fixed25_actual_fraction": 0.5857783914394201, + "fixed25_rescue": 352, + "fixed25_harm": 971, + "rank25_metric": 0.8108387987573352, + "validation_selection": 0.9390833333333334, + "best_epoch": 2, + "seconds": 5.327153205871582 + }, + { + "id": "extra_0006", + "tag": "student_8M", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6829478771142561, + "fixed25_actual_fraction": 0.6049361408353469, + "fixed25_rescue": 358, + "fixed25_harm": 1119, + "rank25_metric": 0.8091128753883328, + "validation_selection": 0.928, + "best_epoch": 1, + "seconds": 5.070952415466309 + }, + { + "id": "formal_0006", + "tag": "target_error", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7433551950293407, + "fixed25_actual_fraction": 0.2604418363824646, + "fixed25_rescue": 132, + "fixed25_harm": 543, + "rank25_metric": 0.7474974111149465, + "validation_selection": 0.9173333333333333, + "best_epoch": 1, + "seconds": 3.2323741912841797 + }, + { + "id": "formal_0009", + "tag": "baseline_100pct", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6829478771142561, + "fixed25_actual_fraction": 0.6049361408353469, + "fixed25_rescue": 358, + "fixed25_harm": 1119, + "rank25_metric": 0.8091128753883328, + "validation_selection": 0.928, + "best_epoch": 1, + "seconds": 2.053396463394165 + }, + { + "id": "formal_0010", + "tag": "drop_confidence", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.8065239903348291, + "fixed25_actual_fraction": 0.24767000345184673, + "fixed25_rescue": 48, + "fixed25_harm": 93, + "rank25_metric": 0.8065239903348291, + "validation_selection": 0.9790833333333333, + "best_epoch": 1, + "seconds": 2.09718656539917 + }, + { + "id": "formal_0011", + "tag": "drop_output", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6950293406972731, + "fixed25_actual_fraction": 0.5472903003106662, + "fixed25_rescue": 349, + "fixed25_harm": 1040, + "rank25_metric": 0.7894373489817053, + "validation_selection": 0.9210833333333334, + "best_epoch": 1, + "seconds": 1.6060831546783447 + }, + { + "id": "formal_0012", + "tag": "representation_128", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 128, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7711425612702796, + "fixed25_actual_fraction": 0.381429064549534, + "fixed25_rescue": 227, + "fixed25_harm": 477, + "rank25_metric": 0.7897825336555057, + "validation_selection": 0.9603333333333334, + "best_epoch": 50, + "seconds": 9.383041143417358 + }, + { + "id": "formal_0013", + "tag": "representation_256", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 256, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6919226786330687, + "fixed25_actual_fraction": 0.5954435623058336, + "fixed25_rescue": 357, + "fixed25_harm": 1066, + "rank25_metric": 0.8098032447359337, + "validation_selection": 0.931, + "best_epoch": 2, + "seconds": 2.16186261177063 + }, + { + "id": "formal_0014", + "tag": "student_1M", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6829478771142561, + "fixed25_actual_fraction": 0.6049361408353469, + "fixed25_rescue": 358, + "fixed25_harm": 1119, + "rank25_metric": 0.8091128753883328, + "validation_selection": 0.928, + "best_epoch": 1, + "seconds": 1.4905903339385986 + }, + { + "id": "formal_0015", + "tag": "student_2M", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6829478771142561, + "fixed25_actual_fraction": 0.6049361408353469, + "fixed25_rescue": 358, + "fixed25_harm": 1119, + "rank25_metric": 0.8091128753883328, + "validation_selection": 0.928, + "best_epoch": 1, + "seconds": 2.0411505699157715 + }, + { + "id": "formal_0016", + "tag": "target_error", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6732827062478426, + "fixed25_actual_fraction": 0.5995857783914395, + "fixed25_rescue": 357, + "fixed25_harm": 1174, + "rank25_metric": 0.7889195719710045, + "validation_selection": 0.9179166666666667, + "best_epoch": 1, + "seconds": 2.198946475982666 + }, + { + "id": "formal_0017", + "tag": "target_four_state", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6829478771142561, + "fixed25_actual_fraction": 0.6049361408353469, + "fixed25_rescue": 358, + "fixed25_harm": 1119, + "rank25_metric": 0.8091128753883328, + "validation_selection": 0.928, + "best_epoch": 1, + "seconds": 5.897979497909546 + }, + { + "id": "formal_0018", + "tag": "harm_cost_2", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6846738004832585, + "fixed25_actual_fraction": 0.6052813255091474, + "fixed25_rescue": 358, + "fixed25_harm": 1109, + "rank25_metric": 0.8096306523990335, + "validation_selection": 0.9307500000000001, + "best_epoch": 1, + "seconds": 5.417069911956787 + }, + { + "id": "formal_0189", + "tag": "baseline_25pct", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8674490852606145, + "fixed25_actual_fraction": 0.24818778046254747, + "fixed25_rescue": 11, + "fixed25_harm": 209, + "rank25_metric": 0.8674490852606145, + "validation_selection": 0.9786666666666667, + "best_epoch": 12, + "seconds": 6.889333963394165 + }, + { + "id": "formal_0190", + "tag": "baseline_50pct", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.870037970314118, + "fixed25_actual_fraction": 0.24991370383154987, + "fixed25_rescue": 13, + "fixed25_harm": 196, + "rank25_metric": 0.870037970314118, + "validation_selection": 0.9773333333333334, + "best_epoch": 6, + "seconds": 5.409059524536133 + }, + { + "id": "extra_0007", + "tag": "representation_32", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 32, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8690024162927166, + "fixed25_actual_fraction": 0.24853296513634796, + "fixed25_rescue": 11, + "fixed25_harm": 200, + "rank25_metric": 0.8686572316189162, + "validation_selection": 0.9784166666666667, + "best_epoch": 15, + "seconds": 10.294649839401245 + }, + { + "id": "extra_0008", + "tag": "representation_64", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 64, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8679668622713151, + "fixed25_actual_fraction": 0.24974111149464964, + "fixed25_rescue": 13, + "fixed25_harm": 208, + "rank25_metric": 0.8679668622713151, + "validation_selection": 0.9781666666666666, + "best_epoch": 5, + "seconds": 5.710233926773071 + }, + { + "id": "extra_0009", + "tag": "student_8M", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8658957542285123, + "fixed25_actual_fraction": 0.25509147393855713, + "fixed25_rescue": 13, + "fixed25_harm": 220, + "rank25_metric": 0.8679668622713151, + "validation_selection": 0.9788333333333333, + "best_epoch": 9, + "seconds": 7.065940618515015 + }, + { + "id": "formal_0019", + "tag": "baseline_100pct", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8658957542285123, + "fixed25_actual_fraction": 0.25509147393855713, + "fixed25_rescue": 13, + "fixed25_harm": 220, + "rank25_metric": 0.8679668622713151, + "validation_selection": 0.9788333333333333, + "best_epoch": 9, + "seconds": 7.6912243366241455 + }, + { + "id": "formal_0020", + "tag": "representation_128", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 128, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8679668622713151, + "fixed25_actual_fraction": 0.24974111149464964, + "fixed25_rescue": 13, + "fixed25_harm": 208, + "rank25_metric": 0.8679668622713151, + "validation_selection": 0.9786666666666667, + "best_epoch": 9, + "seconds": 7.554404973983765 + }, + { + "id": "formal_0021", + "tag": "representation_256", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 256, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.870037970314118, + "fixed25_actual_fraction": 0.24991370383154987, + "fixed25_rescue": 13, + "fixed25_harm": 196, + "rank25_metric": 0.870037970314118, + "validation_selection": 0.9788333333333333, + "best_epoch": 16, + "seconds": 9.600196123123169 + }, + { + "id": "formal_0022", + "tag": "student_1M", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8658957542285123, + "fixed25_actual_fraction": 0.25509147393855713, + "fixed25_rescue": 13, + "fixed25_harm": 220, + "rank25_metric": 0.8679668622713151, + "validation_selection": 0.9788333333333333, + "best_epoch": 9, + "seconds": 7.301284313201904 + }, + { + "id": "formal_0023", + "tag": "student_2M", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8658957542285123, + "fixed25_actual_fraction": 0.25509147393855713, + "fixed25_rescue": 13, + "fixed25_harm": 220, + "rank25_metric": 0.8679668622713151, + "validation_selection": 0.9788333333333333, + "best_epoch": 9, + "seconds": 6.240914821624756 + }, + { + "id": "formal_0024", + "tag": "target_error", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.7664825681739731, + "fixed25_actual_fraction": 0.24767000345184673, + "fixed25_rescue": 46, + "fixed25_harm": 829, + "rank25_metric": 0.7664825681739731, + "validation_selection": 0.9097500000000001, + "best_epoch": 16, + "seconds": 11.102000713348389 + }, + { + "id": "formal_0025", + "tag": "target_four_state", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8683120469451157, + "fixed25_actual_fraction": 0.25008629616845013, + "fixed25_rescue": 16, + "fixed25_harm": 209, + "rank25_metric": 0.8683120469451157, + "validation_selection": 0.9762500000000001, + "best_epoch": 2, + "seconds": 3.639625310897827 + }, + { + "id": "formal_0026", + "tag": "harm_cost_2", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.870037970314118, + "fixed25_actual_fraction": 0.24991370383154987, + "fixed25_rescue": 13, + "fixed25_harm": 196, + "rank25_metric": 0.870037970314118, + "validation_selection": 0.9783333333333333, + "best_epoch": 6, + "seconds": 7.290773391723633 + }, + { + "id": "formal_0038", + "tag": "target_error", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.7580255436658613, + "fixed25_actual_fraction": 0.25232999654815325, + "fixed25_rescue": 49, + "fixed25_harm": 881, + "rank25_metric": 0.7597514670348636, + "validation_selection": 0.9071666666666667, + "best_epoch": 4, + "seconds": 9.048983573913574 + }, + { + "id": "formal_0191", + "tag": "baseline_25pct", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8797031411805316, + "fixed25_actual_fraction": 0.23662409389023129, + "fixed25_rescue": 18, + "fixed25_harm": 145, + "rank25_metric": 0.8776320331377286, + "validation_selection": 0.98025, + "best_epoch": 18, + "seconds": 7.610384702682495 + }, + { + "id": "formal_0192", + "tag": "baseline_50pct", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.876423886779427, + "fixed25_actual_fraction": 0.24525371073524335, + "fixed25_rescue": 8, + "fixed25_harm": 154, + "rank25_metric": 0.8762512944425268, + "validation_selection": 0.981, + "best_epoch": 25, + "seconds": 10.510464906692505 + }, + { + "id": "extra_0010", + "tag": "representation_32", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 32, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8836727649292372, + "fixed25_actual_fraction": 0.24370037970314118, + "fixed25_rescue": 5, + "fixed25_harm": 109, + "rank25_metric": 0.882119433897135, + "validation_selection": 0.9845833333333334, + "best_epoch": 3, + "seconds": 5.182578086853027 + }, + { + "id": "extra_0011", + "tag": "representation_64", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 64, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8855712806351398, + "fixed25_actual_fraction": 0.23662409389023129, + "fixed25_rescue": 5, + "fixed25_harm": 98, + "rank25_metric": 0.8843631342768381, + "validation_selection": 0.9846666666666666, + "best_epoch": 3, + "seconds": 5.3754966259002686 + }, + { + "id": "extra_0012", + "tag": "student_8M", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.870728339661719, + "fixed25_actual_fraction": 0.24853296513634796, + "fixed25_rescue": 14, + "fixed25_harm": 193, + "rank25_metric": 0.8705557473248188, + "validation_selection": 0.977, + "best_epoch": 1, + "seconds": 4.12035346031189 + }, + { + "id": "formal_0024", + "tag": "target_error", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8075595443562306, + "fixed25_actual_fraction": 0.24767000345184673, + "fixed25_rescue": 56, + "fixed25_harm": 601, + "rank25_metric": 0.8070417673455299, + "validation_selection": 0.9256666666666666, + "best_epoch": 16, + "seconds": 11.102000713348389 + }, + { + "id": "formal_0027", + "tag": "baseline_100pct", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8684846392820159, + "fixed25_actual_fraction": 0.2507766655160511, + "fixed25_rescue": 17, + "fixed25_harm": 209, + "rank25_metric": 0.8684846392820159, + "validation_selection": 0.97525, + "best_epoch": 1, + "seconds": 3.935011863708496 + }, + { + "id": "formal_0028", + "tag": "drop_image", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8788401794960303, + "fixed25_actual_fraction": 0.25181221953745253, + "fixed25_rescue": 8, + "fixed25_harm": 140, + "rank25_metric": 0.8793579565067311, + "validation_selection": 0.9799166666666667, + "best_epoch": 23, + "seconds": 13.344428062438965 + }, + { + "id": "formal_0029", + "tag": "drop_text", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8733172247152227, + "fixed25_actual_fraction": 0.2430100103555402, + "fixed25_rescue": 17, + "fixed25_harm": 181, + "rank25_metric": 0.8722816706938212, + "validation_selection": 0.9743333333333333, + "best_epoch": 5, + "seconds": 6.613906621932983 + }, + { + "id": "formal_0030", + "tag": "drop_hidden", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8372454263030722, + "fixed25_actual_fraction": 0.26561960648947186, + "fixed25_rescue": 27, + "fixed25_harm": 400, + "rank25_metric": 0.843631342768381, + "validation_selection": 0.9608333333333333, + "best_epoch": 41, + "seconds": 18.277963161468506 + }, + { + "id": "formal_0031", + "tag": "hidden_position_aggregator__layer_01", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8731446323783224, + "fixed25_actual_fraction": 0.24767000345184673, + "fixed25_rescue": 22, + "fixed25_harm": 187, + "rank25_metric": 0.8727994477045219, + "validation_selection": 0.9765, + "best_epoch": 1, + "seconds": 5.343691825866699 + }, + { + "id": "formal_0032", + "tag": "hidden_position_global_local_encoder__layer_06", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_06", + "fixed25_metric": 0.8419054193993787, + "fixed25_actual_fraction": 0.2575077666551605, + "fixed25_rescue": 36, + "fixed25_harm": 382, + "rank25_metric": 0.8439765274421815, + "validation_selection": 0.9591666666666667, + "best_epoch": 1, + "seconds": 5.476740121841431 + }, + { + "id": "formal_0033", + "tag": "hidden_multi_layer", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8738350017259233, + "fixed25_actual_fraction": 0.24767000345184673, + "fixed25_rescue": 13, + "fixed25_harm": 174, + "rank25_metric": 0.8733172247152227, + "validation_selection": 0.9786666666666667, + "best_epoch": 2, + "seconds": 6.360007286071777 + }, + { + "id": "formal_0034", + "tag": "representation_128", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 128, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8810838798757336, + "fixed25_actual_fraction": 0.25112185018985156, + "fixed25_rescue": 7, + "fixed25_harm": 126, + "rank25_metric": 0.8809112875388333, + "validation_selection": 0.9837499999999999, + "best_epoch": 2, + "seconds": 6.371859073638916 + }, + { + "id": "formal_0035", + "tag": "representation_256", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 256, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8783224024853297, + "fixed25_actual_fraction": 0.2474974111149465, + "fixed25_rescue": 16, + "fixed25_harm": 151, + "rank25_metric": 0.8779772178115292, + "validation_selection": 0.9819166666666667, + "best_epoch": 1, + "seconds": 3.1990182399749756 + }, + { + "id": "formal_0036", + "tag": "student_1M", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.865723161891612, + "fixed25_actual_fraction": 0.25560925094925785, + "fixed25_rescue": 16, + "fixed25_harm": 224, + "rank25_metric": 0.8662409389023127, + "validation_selection": 0.97575, + "best_epoch": 1, + "seconds": 5.600154161453247 + }, + { + "id": "formal_0037", + "tag": "student_2M", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8662409389023127, + "fixed25_actual_fraction": 0.2659647911632724, + "fixed25_rescue": 19, + "fixed25_harm": 224, + "rank25_metric": 0.8683120469451157, + "validation_selection": 0.97625, + "best_epoch": 1, + "seconds": 5.593548774719238 + }, + { + "id": "formal_0038", + "tag": "target_error", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8282706247842596, + "fixed25_actual_fraction": 0.24197445633413878, + "fixed25_rescue": 40, + "fixed25_harm": 465, + "rank25_metric": 0.8260269244045564, + "validation_selection": 0.94, + "best_epoch": 7, + "seconds": 9.048983573913574 + }, + { + "id": "formal_0039", + "tag": "target_four_state", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8684846392820159, + "fixed25_actual_fraction": 0.2507766655160511, + "fixed25_rescue": 17, + "fixed25_harm": 209, + "rank25_metric": 0.8684846392820159, + "validation_selection": 0.97525, + "best_epoch": 1, + "seconds": 5.580618619918823 + }, + { + "id": "formal_0040", + "tag": "fusion_attention", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8669313082499137, + "fixed25_actual_fraction": 0.2597514670348636, + "fixed25_rescue": 14, + "fixed25_harm": 215, + "rank25_metric": 0.8684846392820159, + "validation_selection": 0.9769166666666667, + "best_epoch": 1, + "seconds": 7.3831634521484375 + }, + { + "id": "formal_0041", + "tag": "harm_cost_2", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8717638936831205, + "fixed25_actual_fraction": 0.2599240593717639, + "fixed25_rescue": 13, + "fixed25_harm": 186, + "rank25_metric": 0.8729720400414221, + "validation_selection": 0.9796666666666667, + "best_epoch": 1, + "seconds": 3.963697910308838 + }, + { + "id": "formal_0193", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.38158976831239544, + "fixed25_actual_fraction": 0.24836230609987806, + "fixed25_rescue": 4083, + "fixed25_harm": 38, + "rank25_metric": 0.38201514335138814, + "validation_selection": 0.6237472766884531, + "best_epoch": 1, + "seconds": 28.353870153427124 + }, + { + "id": "formal_0194", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3658792502055979, + "fixed25_actual_fraction": 0.22034427019822478, + "fixed25_rescue": 3503, + "fixed25_harm": 12, + "rank25_metric": 0.3758330261180274, + "validation_selection": 0.6238526951999438, + "best_epoch": 2, + "seconds": 56.78019833564758 + }, + { + "id": "extra_0013", + "tag": "representation_32", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3701613589314579, + "fixed25_actual_fraction": 0.2273487791736381, + "fixed25_rescue": 3680, + "fixed25_harm": 38, + "rank25_metric": 0.37838527635198366, + "validation_selection": 0.6219270503900485, + "best_epoch": 2, + "seconds": 128.10148692131042 + }, + { + "id": "extra_0014", + "tag": "representation_64", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3667583586195162, + "fixed25_actual_fraction": 0.22235771204945695, + "fixed25_rescue": 3554, + "fixed25_harm": 32, + "rank25_metric": 0.37668377619601284, + "validation_selection": 0.6208728652751423, + "best_epoch": 2, + "seconds": 127.02403688430786 + }, + { + "id": "extra_0015", + "tag": "student_8M", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.37294047585287693, + "fixed25_actual_fraction": 0.2379264384765902, + "fixed25_rescue": 3803, + "fixed25_harm": 63, + "rank25_metric": 0.378130051328588, + "validation_selection": 0.6220605805046033, + "best_epoch": 1, + "seconds": 114.63445687294006 + }, + { + "id": "formal_0042", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3693106088534725, + "fixed25_actual_fraction": 0.22853982928281769, + "fixed25_rescue": 3642, + "fixed25_harm": 30, + "rank25_metric": 0.3758330261180274, + "validation_selection": 0.6221730269168599, + "best_epoch": 9, + "seconds": 476.6986792087555 + }, + { + "id": "formal_0043", + "tag": "drop_image", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.364376258401157, + "fixed25_actual_fraction": 0.21804724498766412, + "fixed25_rescue": 3447, + "fixed25_harm": 9, + "rank25_metric": 0.3735643592433996, + "validation_selection": 0.6235926628716002, + "best_epoch": 1, + "seconds": 195.11448216438293 + }, + { + "id": "formal_0044", + "tag": "drop_text", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.37495391770410913, + "fixed25_actual_fraction": 0.2399115219918895, + "fixed25_rescue": 3854, + "fixed25_harm": 43, + "rank25_metric": 0.3789240847347078, + "validation_selection": 0.6211891208096142, + "best_epoch": 1, + "seconds": 261.2588014602661 + }, + { + "id": "formal_0045", + "tag": "drop_hidden", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.36732552533817314, + "fixed25_actual_fraction": 0.22170547032300145, + "fixed25_rescue": 3589, + "fixed25_harm": 47, + "rank25_metric": 0.37725094291466976, + "validation_selection": 0.6216881017640031, + "best_epoch": 7, + "seconds": 358.06045365333557 + }, + { + "id": "formal_0046", + "tag": "drop_confidence", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.36247624989365623, + "fixed25_actual_fraction": 0.2210248702606131, + "fixed25_rescue": 3395, + "fixed25_harm": 24, + "rank25_metric": 0.372741967501347, + "validation_selection": 0.6210907301988895, + "best_epoch": 1, + "seconds": 263.48469948768616 + }, + { + "id": "formal_0047", + "tag": "hidden_position_multimodal_encoder__layer_01", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_01", + "fixed25_metric": 0.3823270850466495, + "fixed25_actual_fraction": 0.24632050591271304, + "fixed25_rescue": 4165, + "fixed25_harm": 94, + "rank25_metric": 0.38365992683549327, + "validation_selection": 0.6220535526038372, + "best_epoch": 7, + "seconds": 426.6365387439728 + }, + { + "id": "formal_0048", + "tag": "hidden_position_text_encoder__layer_06", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_06", + "fixed25_metric": 0.3687434421348155, + "fixed25_actual_fraction": 0.22797266256416074, + "fixed25_rescue": 3629, + "fixed25_harm": 37, + "rank25_metric": 0.3761449678132887, + "validation_selection": 0.6237964719938154, + "best_epoch": 3, + "seconds": 326.0554850101471 + }, + { + "id": "formal_0049", + "tag": "hidden_multi_layer", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_01", + "fixed25_metric": 0.3881121855769503, + "fixed25_actual_fraction": 0.2539205399427162, + "fixed25_rescue": 4332, + "fixed25_harm": 57, + "rank25_metric": 0.38638232708504666, + "validation_selection": 0.6241478670321174, + "best_epoch": 3, + "seconds": 315.55904817581177 + }, + { + "id": "formal_0050", + "tag": "representation_128", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3645180500808213, + "fixed25_actual_fraction": 0.21955023679210503, + "fixed25_rescue": 3463, + "fixed25_harm": 20, + "rank25_metric": 0.374046450954258, + "validation_selection": 0.6219903014969428, + "best_epoch": 4, + "seconds": 351.69083309173584 + }, + { + "id": "formal_0051", + "tag": "representation_256", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3698777755721294, + "fixed25_actual_fraction": 0.2289652043218104, + "fixed25_rescue": 3671, + "fixed25_harm": 39, + "rank25_metric": 0.3778748263051924, + "validation_selection": 0.621856771382388, + "best_epoch": 3, + "seconds": 296.8873279094696 + }, + { + "id": "formal_0052", + "tag": "student_1M", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3665314919320534, + "fixed25_actual_fraction": 0.22278308708844966, + "fixed25_rescue": 3530, + "fixed25_harm": 16, + "rank25_metric": 0.3759464594617588, + "validation_selection": 0.6230093471080188, + "best_epoch": 1, + "seconds": 240.49552273750305 + }, + { + "id": "formal_0053", + "tag": "student_2M", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.37197629243116015, + "fixed25_actual_fraction": 0.23001446275132575, + "fixed25_rescue": 3743, + "fixed25_harm": 37, + "rank25_metric": 0.38022856818761874, + "validation_selection": 0.6234872443601096, + "best_epoch": 3, + "seconds": 289.043110370636 + }, + { + "id": "formal_0054", + "tag": "target_error", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.37489720103224344, + "fixed25_actual_fraction": 0.24345631398349546, + "fixed25_rescue": 3872, + "fixed25_harm": 63, + "rank25_metric": 0.37705243456313986, + "validation_selection": 0.6214210415348935, + "best_epoch": 1, + "seconds": 351.4256999492645 + }, + { + "id": "formal_0055", + "tag": "target_four_state", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.37872557638317783, + "fixed25_actual_fraction": 0.23265178799308056, + "fixed25_rescue": 4019, + "fixed25_harm": 75, + "rank25_metric": 0.38530471031959845, + "validation_selection": 0.6250544662309369, + "best_epoch": 4, + "seconds": 317.04318404197693 + }, + { + "id": "formal_0056", + "tag": "fusion_concat", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.36426282505742563, + "fixed25_actual_fraction": 0.21901142840938093, + "fixed25_rescue": 3456, + "fixed25_harm": 22, + "rank25_metric": 0.37401809261832514, + "validation_selection": 0.6226157846651205, + "best_epoch": 4, + "seconds": 275.1700701713562 + }, + { + "id": "formal_0069", + "tag": "target_error", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.37370615092306386, + "fixed25_actual_fraction": 0.23596971329722372, + "fixed25_rescue": 3814, + "fixed25_harm": 47, + "rank25_metric": 0.379065876414372, + "validation_selection": 0.6227985100850376, + "best_epoch": 1, + "seconds": 256.6061637401581 + }, + { + "id": "formal_0195", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4562572668235828, + "fixed25_actual_fraction": 0.24856081445140799, + "fixed25_rescue": 6732, + "fixed25_harm": 54, + "rank25_metric": 0.45705130022970253, + "validation_selection": 0.6656265373532926, + "best_epoch": 1, + "seconds": 18.048375368118286 + }, + { + "id": "formal_0196", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.434648214842753, + "fixed25_actual_fraction": 0.22008904517482913, + "fixed25_rescue": 5930, + "fixed25_harm": 14, + "rank25_metric": 0.4536766582536937, + "validation_selection": 0.6660130718954248, + "best_epoch": 2, + "seconds": 33.53883171081543 + }, + { + "id": "extra_0016", + "tag": "representation_32", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4426169072398832, + "fixed25_actual_fraction": 0.2276607208688994, + "fixed25_rescue": 6227, + "fixed25_harm": 30, + "rank25_metric": 0.4566826418625755, + "validation_selection": 0.6649307751774545, + "best_epoch": 2, + "seconds": 85.932044506073 + }, + { + "id": "extra_0017", + "tag": "representation_64", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.44343929898193574, + "fixed25_actual_fraction": 0.22851147094688484, + "fixed25_rescue": 6257, + "fixed25_harm": 31, + "rank25_metric": 0.45719309190936674, + "validation_selection": 0.6650010541851149, + "best_epoch": 2, + "seconds": 86.59067559242249 + }, + { + "id": "extra_0018", + "tag": "student_8M", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.42920341434364634, + "fixed25_actual_fraction": 0.20959646087967557, + "fixed25_rescue": 5739, + "fixed25_harm": 15, + "rank25_metric": 0.4557468167767915, + "validation_selection": 0.6653032539180547, + "best_epoch": 2, + "seconds": 87.00826907157898 + }, + { + "id": "formal_0054", + "tag": "target_error", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4345631398349545, + "fixed25_actual_fraction": 0.2188696367297167, + "fixed25_rescue": 5928, + "fixed25_harm": 15, + "rank25_metric": 0.4539318832770893, + "validation_selection": 0.6653594771241831, + "best_epoch": 4, + "seconds": 351.4256999492645 + }, + { + "id": "formal_0057", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.43033774778096023, + "fixed25_actual_fraction": 0.2126591611604231, + "fixed25_rescue": 5784, + "fixed25_harm": 20, + "rank25_metric": 0.4544139749879477, + "validation_selection": 0.6650924168950734, + "best_epoch": 5, + "seconds": 281.1101198196411 + }, + { + "id": "formal_0058", + "tag": "drop_image", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.43430791481155884, + "fixed25_actual_fraction": 0.21804724498766412, + "fixed25_rescue": 5914, + "fixed25_harm": 10, + "rank25_metric": 0.4541871083004849, + "validation_selection": 0.6667510014758591, + "best_epoch": 1, + "seconds": 166.40395164489746 + }, + { + "id": "formal_0059", + "tag": "drop_text", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.441709440490032, + "fixed25_actual_fraction": 0.22870997929841477, + "fixed25_rescue": 6177, + "fixed25_harm": 12, + "rank25_metric": 0.4553214417377988, + "validation_selection": 0.6643755710169372, + "best_epoch": 2, + "seconds": 181.88760209083557 + }, + { + "id": "formal_0060", + "tag": "drop_hidden", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.44636020758301903, + "fixed25_actual_fraction": 0.23795479681252305, + "fixed25_rescue": 6388, + "fixed25_harm": 59, + "rank25_metric": 0.4537333749255594, + "validation_selection": 0.662126642771804, + "best_epoch": 1, + "seconds": 106.5097143650055 + }, + { + "id": "formal_0061", + "tag": "drop_confidence", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4283526642656609, + "fixed25_actual_fraction": 0.21180841108243767, + "fixed25_rescue": 5705, + "fixed25_harm": 11, + "rank25_metric": 0.45384680826929075, + "validation_selection": 0.6646215475437487, + "best_epoch": 2, + "seconds": 186.71321535110474 + }, + { + "id": "formal_0062", + "tag": "hidden_position_multimodal_encoder__layer_01", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_01", + "fixed25_metric": 0.45812891699515074, + "fixed25_actual_fraction": 0.2401383886793523, + "fixed25_rescue": 6809, + "fixed25_harm": 65, + "rank25_metric": 0.4648782009471684, + "validation_selection": 0.6651205284981376, + "best_epoch": 7, + "seconds": 283.8257784843445 + }, + { + "id": "formal_0063", + "tag": "hidden_position_text_encoder__layer_06", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_06", + "fixed25_metric": 0.44017809034965827, + "fixed25_actual_fraction": 0.22984431273572867, + "fixed25_rescue": 6150, + "fixed25_harm": 39, + "rank25_metric": 0.452684116496044, + "validation_selection": 0.6649096914751563, + "best_epoch": 7, + "seconds": 320.17189478874207 + }, + { + "id": "formal_0064", + "tag": "hidden_multi_layer", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_01", + "fixed25_metric": 0.4681394095794459, + "fixed25_actual_fraction": 0.25074440631823725, + "fixed25_rescue": 7146, + "fixed25_harm": 49, + "rank25_metric": 0.4675155261889232, + "validation_selection": 0.6663012158268324, + "best_epoch": 3, + "seconds": 237.93119263648987 + }, + { + "id": "formal_0065", + "tag": "representation_128", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.43632135666279104, + "fixed25_actual_fraction": 0.22091143691688173, + "fixed25_rescue": 5996, + "fixed25_harm": 21, + "rank25_metric": 0.4532796415506338, + "validation_selection": 0.6649026635743903, + "best_epoch": 4, + "seconds": 234.7888114452362 + }, + { + "id": "formal_0066", + "tag": "representation_256", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.43663329835805237, + "fixed25_actual_fraction": 0.21872784505005247, + "fixed25_rescue": 6003, + "fixed25_harm": 17, + "rank25_metric": 0.4554348750815302, + "validation_selection": 0.6652259470096282, + "best_epoch": 4, + "seconds": 231.40922713279724 + }, + { + "id": "formal_0067", + "tag": "student_1M", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.43731389842044066, + "fixed25_actual_fraction": 0.22224427870572555, + "fixed25_rescue": 6021, + "fixed25_harm": 11, + "rank25_metric": 0.4541020332926864, + "validation_selection": 0.6657179000632512, + "best_epoch": 4, + "seconds": 229.5956084728241 + }, + { + "id": "formal_0068", + "tag": "student_2M", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4318123812494683, + "fixed25_actual_fraction": 0.21399200294926693, + "fixed25_rescue": 5830, + "fixed25_harm": 14, + "rank25_metric": 0.4548109916910076, + "validation_selection": 0.6654157003303113, + "best_epoch": 2, + "seconds": 199.25073266029358 + }, + { + "id": "formal_0069", + "tag": "target_error", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4454810991691008, + "fixed25_actual_fraction": 0.23177267957916228, + "fixed25_rescue": 6322, + "fixed25_harm": 24, + "rank25_metric": 0.4573065252530982, + "validation_selection": 0.6649659146812846, + "best_epoch": 5, + "seconds": 256.6061637401581 + }, + { + "id": "formal_0070", + "tag": "target_four_state", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.438561665201486, + "fixed25_actual_fraction": 0.21575021977710349, + "fixed25_rescue": 6099, + "fixed25_harm": 45, + "rank25_metric": 0.4611632589399654, + "validation_selection": 0.6686977299880525, + "best_epoch": 7, + "seconds": 314.07090616226196 + }, + { + "id": "formal_0071", + "tag": "fusion_attention", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4318123812494683, + "fixed25_actual_fraction": 0.2146158863397896, + "fixed25_rescue": 5831, + "fixed25_harm": 15, + "rank25_metric": 0.4541587499645521, + "validation_selection": 0.665190807505798, + "best_epoch": 4, + "seconds": 340.09505891799927 + }, + { + "id": "formal_0197", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6523268014632901, + "fixed25_actual_fraction": 0.3479284235601055, + "fixed25_rescue": 1394, + "fixed25_harm": 2000, + "rank25_metric": 0.6652582026486685, + "validation_selection": 0.753925082577834, + "best_epoch": 13, + "seconds": 28.585034132003784 + }, + { + "id": "formal_0198", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6567223435328815, + "fixed25_actual_fraction": 0.3123670703003148, + "fixed25_rescue": 1366, + "fixed25_harm": 1817, + "rank25_metric": 0.662762669086578, + "validation_selection": 0.7576779815868999, + "best_epoch": 13, + "seconds": 44.9037389755249 + }, + { + "id": "extra_0019", + "tag": "representation_32", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6564104018376201, + "fixed25_actual_fraction": 0.32365368800158806, + "fixed25_rescue": 1502, + "fixed25_harm": 1964, + "rank25_metric": 0.6659671610469897, + "validation_selection": 0.7572914470447678, + "best_epoch": 3, + "seconds": 78.71963858604431 + }, + { + "id": "extra_0020", + "tag": "representation_64", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6560417434704932, + "fixed25_actual_fraction": 0.318464112525877, + "fixed25_rescue": 1439, + "fixed25_harm": 1914, + "rank25_metric": 0.6641522275472875, + "validation_selection": 0.7562864572352239, + "best_epoch": 3, + "seconds": 77.79833817481995 + }, + { + "id": "extra_0021", + "tag": "student_8M", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.654765618353515, + "fixed25_actual_fraction": 0.33953435612398264, + "fixed25_rescue": 1510, + "fixed25_harm": 2030, + "rank25_metric": 0.6655985026798628, + "validation_selection": 0.7589219200224893, + "best_epoch": 3, + "seconds": 79.3776524066925 + }, + { + "id": "formal_0072", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.654765618353515, + "fixed25_actual_fraction": 0.33953435612398264, + "fixed25_rescue": 1510, + "fixed25_harm": 2030, + "rank25_metric": 0.6655985026798628, + "validation_selection": 0.7589219200224893, + "best_epoch": 3, + "seconds": 163.15981340408325 + }, + { + "id": "formal_0073", + "tag": "hidden_position_joint_encoder__layer_02", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "joint_encoder__layer_02", + "fixed25_metric": 0.6435924339959731, + "fixed25_actual_fraction": 0.28193857584436943, + "fixed25_rescue": 877, + "fixed25_harm": 1791, + "rank25_metric": 0.6475626010265718, + "validation_selection": 0.7462927823459133, + "best_epoch": 7, + "seconds": 218.39065027236938 + }, + { + "id": "formal_0074", + "tag": "hidden_position_lateral_features__layer_02", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_02", + "fixed25_metric": 0.6621671440319882, + "fixed25_actual_fraction": 0.2838669426878031, + "fixed25_rescue": 1361, + "fixed25_harm": 1620, + "rank25_metric": 0.665740294359527, + "validation_selection": 0.7524351676154333, + "best_epoch": 6, + "seconds": 191.70505738258362 + }, + { + "id": "formal_0075", + "tag": "hidden_multi_layer", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "joint_encoder__layer_02", + "fixed25_metric": 0.6581686186654567, + "fixed25_actual_fraction": 0.30692226980120807, + "fixed25_rescue": 1428, + "fixed25_harm": 1828, + "rank25_metric": 0.6646059609222131, + "validation_selection": 0.7494834492936959, + "best_epoch": 3, + "seconds": 145.08960509300232 + }, + { + "id": "formal_0076", + "tag": "representation_128", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6579984686498597, + "fixed25_actual_fraction": 0.3205626293849077, + "fixed25_rescue": 1415, + "fixed25_harm": 1821, + "rank25_metric": 0.6648895442815416, + "validation_selection": 0.7587883899079345, + "best_epoch": 3, + "seconds": 146.03866457939148 + }, + { + "id": "formal_0077", + "tag": "representation_256", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6572327935796728, + "fixed25_actual_fraction": 0.32413577971244645, + "fixed25_rescue": 1438, + "fixed25_harm": 1871, + "rank25_metric": 0.6655701443439299, + "validation_selection": 0.7598074355190104, + "best_epoch": 3, + "seconds": 144.2073380947113 + }, + { + "id": "formal_0078", + "tag": "student_1M", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.654765618353515, + "fixed25_actual_fraction": 0.33953435612398264, + "fixed25_rescue": 1510, + "fixed25_harm": 2030, + "rank25_metric": 0.6655985026798628, + "validation_selection": 0.7589219200224893, + "best_epoch": 3, + "seconds": 143.30026388168335 + }, + { + "id": "formal_0079", + "tag": "student_2M", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.654765618353515, + "fixed25_actual_fraction": 0.33953435612398264, + "fixed25_rescue": 1510, + "fixed25_harm": 2030, + "rank25_metric": 0.6655985026798628, + "validation_selection": 0.7589219200224893, + "best_epoch": 3, + "seconds": 163.39957666397095 + }, + { + "id": "formal_0080", + "tag": "target_error", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6521566514476931, + "fixed25_actual_fraction": 0.30961631171482856, + "fixed25_rescue": 1379, + "fixed25_harm": 1991, + "rank25_metric": 0.6618835606726597, + "validation_selection": 0.7524070560123691, + "best_epoch": 7, + "seconds": 200.08106350898743 + }, + { + "id": "formal_0081", + "tag": "target_four_state", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6556730851033661, + "fixed25_actual_fraction": 0.31480588719053965, + "fixed25_rescue": 1509, + "fixed25_harm": 1997, + "rank25_metric": 0.6647477526018774, + "validation_selection": 0.7561529271206691, + "best_epoch": 7, + "seconds": 189.60577154159546 + }, + { + "id": "formal_0082", + "tag": "harm_cost_2", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.659416385446502, + "fixed25_actual_fraction": 0.30834018659785045, + "fixed25_rescue": 1264, + "fixed25_harm": 1620, + "rank25_metric": 0.6656835776876613, + "validation_selection": 0.7544029798299249, + "best_epoch": 3, + "seconds": 147.022536277771 + }, + { + "id": "formal_0088", + "tag": "target_error", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.6762612369906134, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 493, + "fixed25_harm": 255, + "rank25_metric": 0.6638686441879591, + "validation_selection": 0.7442125237191651, + "best_epoch": 2, + "seconds": 150.90026092529297 + }, + { + "id": "formal_0199", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7019538893457732, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 14.837297677993774 + }, + { + "id": "formal_0200", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7416555596517597, + "fixed25_actual_fraction": 0.4823752942177353, + "fixed25_rescue": 3418, + "fixed25_harm": 874, + "rank25_metric": 0.7207838244051838, + "validation_selection": 0.8144985592803429, + "best_epoch": 1, + "seconds": 21.554462432861328 + }, + { + "id": "extra_0022", + "tag": "representation_32", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 32, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7019538893457732, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 70.20556235313416 + }, + { + "id": "extra_0023", + "tag": "representation_64", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 64, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7019538893457732, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 71.17191433906555 + }, + { + "id": "extra_0024", + "tag": "student_8M", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7090151149930523, + "fixed25_actual_fraction": 0.12330204463602076, + "fixed25_rescue": 1514, + "fixed25_harm": 121, + "rank25_metric": 0.7021523976973031, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 71.64453220367432 + }, + { + "id": "formal_0080", + "tag": "target_error", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.7439525848623203, + "fixed25_actual_fraction": 0.3155715622607265, + "fixed25_rescue": 3506, + "fixed25_harm": 881, + "rank25_metric": 0.7360689674729887, + "validation_selection": 0.7762878628153771, + "best_epoch": 5, + "seconds": 200.08106350898743 + }, + { + "id": "formal_0083", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7090151149930523, + "fixed25_actual_fraction": 0.12330204463602076, + "fixed25_rescue": 1514, + "fixed25_harm": 121, + "rank25_metric": 0.7021523976973031, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 121.36263680458069 + }, + { + "id": "formal_0084", + "tag": "representation_128", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 128, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7090151149930523, + "fixed25_actual_fraction": 0.12330204463602076, + "fixed25_rescue": 1514, + "fixed25_harm": 121, + "rank25_metric": 0.7021523976973031, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 118.60358738899231 + }, + { + "id": "formal_0085", + "tag": "representation_256", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 256, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7019538893457732, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 122.39959692955017 + }, + { + "id": "formal_0086", + "tag": "student_1M", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7090151149930523, + "fixed25_actual_fraction": 0.12330204463602076, + "fixed25_rescue": 1514, + "fixed25_harm": 121, + "rank25_metric": 0.7021523976973031, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 120.20404839515686 + }, + { + "id": "formal_0087", + "tag": "student_2M", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7090151149930523, + "fixed25_actual_fraction": 0.12330204463602076, + "fixed25_rescue": 1514, + "fixed25_harm": 121, + "rank25_metric": 0.7021523976973031, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 117.77066564559937 + }, + { + "id": "formal_0088", + "tag": "target_error", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7162748489918611, + "validation_selection": 0.7588446131140628, + "best_epoch": 1, + "seconds": 150.90026092529297 + }, + { + "id": "formal_0089", + "tag": "target_four_state", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7019538893457732, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 129.93259263038635 + }, + { + "id": "formal_0090", + "tag": "harm_cost_2", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7019538893457732, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 129.64483284950256 + }, + { + "id": "formal_0201", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.863815742268781, + "fixed25_actual_fraction": 0.253928392049939, + "fixed25_rescue": 1100, + "fixed25_harm": 211, + "rank25_metric": 0.8630264762861448, + "validation_selection": 0.8331853337152678, + "best_epoch": 2, + "seconds": 11.065314054489136 + }, + { + "id": "formal_0202", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.868336083805697, + "fixed25_actual_fraction": 0.24388318863456984, + "fixed25_rescue": 1152, + "fixed25_harm": 200, + "rank25_metric": 0.8689818468823993, + "validation_selection": 0.8330850759094816, + "best_epoch": 3, + "seconds": 16.297266960144043 + }, + { + "id": "extra_0025", + "tag": "representation_32", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 32, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8694841070531678, + "fixed25_actual_fraction": 0.2631125780297051, + "fixed25_rescue": 1175, + "fixed25_harm": 207, + "rank25_metric": 0.8678338236349286, + "validation_selection": 0.8335147522199943, + "best_epoch": 6, + "seconds": 62.19458746910095 + }, + { + "id": "extra_0026", + "tag": "representation_64", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 64, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8696276099591017, + "fixed25_actual_fraction": 0.26296907512377127, + "fixed25_rescue": 1176, + "fixed25_harm": 206, + "rank25_metric": 0.8669728061993255, + "validation_selection": 0.833887138355772, + "best_epoch": 6, + "seconds": 62.55568766593933 + }, + { + "id": "extra_0027", + "tag": "student_8M", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8707756332065725, + "fixed25_actual_fraction": 0.26117528879959817, + "fixed25_rescue": 1197, + "fixed25_harm": 211, + "rank25_metric": 0.8689100954294324, + "validation_selection": 0.8335577198510455, + "best_epoch": 6, + "seconds": 61.909870862960815 + }, + { + "id": "formal_0091", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8707756332065725, + "fixed25_actual_fraction": 0.26117528879959817, + "fixed25_rescue": 1197, + "fixed25_harm": 211, + "rank25_metric": 0.8689100954294324, + "validation_selection": 0.8335577198510455, + "best_epoch": 6, + "seconds": 82.51407408714294 + }, + { + "id": "formal_0092", + "tag": "drop_hidden", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8648920140632848, + "fixed25_actual_fraction": 0.23871708402095143, + "fixed25_rescue": 1173, + "fixed25_harm": 269, + "rank25_metric": 0.8675468178230609, + "validation_selection": 0.8284445717559438, + "best_epoch": 2, + "seconds": 54.881943464279175 + }, + { + "id": "formal_0093", + "tag": "drop_confidence", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8552055679127503, + "fixed25_actual_fraction": 0.2546459065796082, + "fixed25_rescue": 946, + "fixed25_harm": 177, + "rank25_metric": 0.8551338164597833, + "validation_selection": 0.827427671154397, + "best_epoch": 6, + "seconds": 73.42077493667603 + }, + { + "id": "formal_0094", + "tag": "hidden_position_multimodal_encoder__layer_02", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_02", + "fixed25_metric": 0.8604434239793356, + "fixed25_actual_fraction": 0.243739685728636, + "fixed25_rescue": 1079, + "fixed25_harm": 237, + "rank25_metric": 0.8617349501327402, + "validation_selection": 0.8337439129189345, + "best_epoch": 21, + "seconds": 164.92656469345093 + }, + { + "id": "formal_0095", + "tag": "hidden_position_multimodal_encoder__layer_08", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_08", + "fixed25_metric": 0.8613761928679056, + "fixed25_actual_fraction": 0.2468967496591806, + "fixed25_rescue": 1098, + "fixed25_harm": 243, + "rank25_metric": 0.8616631986797733, + "validation_selection": 0.8295474076195932, + "best_epoch": 1, + "seconds": 62.42424654960632 + }, + { + "id": "formal_0096", + "tag": "hidden_multi_layer", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_02", + "fixed25_metric": 0.8651072684221855, + "fixed25_actual_fraction": 0.2501255650426921, + "fixed25_rescue": 1132, + "fixed25_harm": 225, + "rank25_metric": 0.8650355169692187, + "validation_selection": 0.8327413348610714, + "best_epoch": 1, + "seconds": 56.55902671813965 + }, + { + "id": "formal_0097", + "tag": "representation_128", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 128, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8690535983353663, + "fixed25_actual_fraction": 0.2578747219631198, + "fixed25_rescue": 1175, + "fixed25_harm": 213, + "rank25_metric": 0.8676903207289948, + "validation_selection": 0.8337582354626182, + "best_epoch": 6, + "seconds": 81.54244780540466 + }, + { + "id": "formal_0098", + "tag": "representation_256", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 256, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8663270431226232, + "fixed25_actual_fraction": 0.24933629906005597, + "fixed25_rescue": 1131, + "fixed25_harm": 207, + "rank25_metric": 0.8664705460285571, + "validation_selection": 0.8330134631910627, + "best_epoch": 3, + "seconds": 64.1797993183136 + }, + { + "id": "formal_0099", + "tag": "student_1M", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8707756332065725, + "fixed25_actual_fraction": 0.26117528879959817, + "fixed25_rescue": 1197, + "fixed25_harm": 211, + "rank25_metric": 0.8689100954294324, + "validation_selection": 0.8335577198510455, + "best_epoch": 6, + "seconds": 85.27223777770996 + }, + { + "id": "formal_0100", + "tag": "student_2M", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8707756332065725, + "fixed25_actual_fraction": 0.26117528879959817, + "fixed25_rescue": 1197, + "fixed25_harm": 211, + "rank25_metric": 0.8689100954294324, + "validation_selection": 0.8335577198510455, + "best_epoch": 6, + "seconds": 90.53286790847778 + }, + { + "id": "formal_0101", + "tag": "target_error", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8702016215828371, + "fixed25_actual_fraction": 0.2504125708545598, + "fixed25_rescue": 1230, + "fixed25_harm": 252, + "rank25_metric": 0.8702016215828371, + "validation_selection": 0.8348324262389, + "best_epoch": 6, + "seconds": 89.40919137001038 + }, + { + "id": "formal_0102", + "tag": "target_four_state", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8702016215828371, + "fixed25_actual_fraction": 0.2523498600846667, + "fixed25_rescue": 1206, + "fixed25_harm": 228, + "rank25_metric": 0.8700581186769032, + "validation_selection": 0.8349470065883701, + "best_epoch": 11, + "seconds": 110.4758677482605 + }, + { + "id": "formal_0103", + "tag": "harm_cost_2", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8582908803903279, + "fixed25_actual_fraction": 0.2504843223075267, + "fixed25_rescue": 985, + "fixed25_harm": 173, + "rank25_metric": 0.8582908803903279, + "validation_selection": 0.8309510169006016, + "best_epoch": 13, + "seconds": 114.95233988761902 + }, + { + "id": "formal_0111", + "tag": "target_error", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8696276099591017, + "fixed25_actual_fraction": 0.24926454760708905, + "fixed25_rescue": 1235, + "fixed25_harm": 265, + "rank25_metric": 0.8697711128650355, + "validation_selection": 0.8327986250358064, + "best_epoch": 7, + "seconds": 75.30136489868164 + }, + { + "id": "formal_0203", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8717801535481093, + "fixed25_actual_fraction": 0.25299562316136903, + "fixed25_rescue": 1180, + "fixed25_harm": 180, + "rank25_metric": 0.8714213962832748, + "validation_selection": 0.8382268690919508, + "best_epoch": 3, + "seconds": 12.40796971321106 + }, + { + "id": "formal_0204", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.872067159359977, + "fixed25_actual_fraction": 0.25407189495587285, + "fixed25_rescue": 1178, + "fixed25_harm": 174, + "rank25_metric": 0.8703451244887709, + "validation_selection": 0.8385276425093097, + "best_epoch": 11, + "seconds": 27.56421136856079 + }, + { + "id": "extra_0028", + "tag": "representation_32", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8714213962832748, + "fixed25_actual_fraction": 0.2545024036736744, + "fixed25_rescue": 1163, + "fixed25_harm": 168, + "rank25_metric": 0.8705603788476717, + "validation_selection": 0.838269836723002, + "best_epoch": 1, + "seconds": 50.37446331977844 + }, + { + "id": "extra_0029", + "tag": "representation_64", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.871206141924374, + "fixed25_actual_fraction": 0.2544306522207075, + "fixed25_rescue": 1158, + "fixed25_harm": 166, + "rank25_metric": 0.8703451244887709, + "validation_selection": 0.8382268690919508, + "best_epoch": 7, + "seconds": 70.82060599327087 + }, + { + "id": "extra_0030", + "tag": "student_8M", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8709191361125063, + "fixed25_actual_fraction": 0.25400014350290595, + "fixed25_rescue": 1153, + "fixed25_harm": 165, + "rank25_metric": 0.8695558585061347, + "validation_selection": 0.8383414494414207, + "best_epoch": 7, + "seconds": 70.37148523330688 + }, + { + "id": "formal_0101", + "tag": "target_error", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8707756332065725, + "fixed25_actual_fraction": 0.2504125708545598, + "fixed25_rescue": 1205, + "fixed25_harm": 219, + "rank25_metric": 0.8707038817536055, + "validation_selection": 0.8368375823546261, + "best_epoch": 6, + "seconds": 89.40919137001038 + }, + { + "id": "formal_0104", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8709191361125063, + "fixed25_actual_fraction": 0.25400014350290595, + "fixed25_rescue": 1153, + "fixed25_harm": 165, + "rank25_metric": 0.8695558585061347, + "validation_selection": 0.8383414494414207, + "best_epoch": 7, + "seconds": 91.79227209091187 + }, + { + "id": "formal_0105", + "tag": "drop_confidence", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8000287005811868, + "fixed25_actual_fraction": 0.0, + "fixed25_rescue": 0, + "fixed25_harm": 0, + "rank25_metric": 0.8355456697998135, + "validation_selection": 0.8181323403036379, + "best_epoch": 1, + "seconds": 47.980828046798706 + }, + { + "id": "formal_0106", + "tag": "drop_output", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8656812800459209, + "fixed25_actual_fraction": 0.23871708402095143, + "fixed25_rescue": 1186, + "fixed25_harm": 271, + "rank25_metric": 0.8671880605582263, + "validation_selection": 0.8313806932111143, + "best_epoch": 1, + "seconds": 45.731327533721924 + }, + { + "id": "formal_0107", + "tag": "representation_128", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8709191361125063, + "fixed25_actual_fraction": 0.2530673746143359, + "fixed25_rescue": 1155, + "fixed25_harm": 167, + "rank25_metric": 0.8704168759417378, + "validation_selection": 0.8383128043540533, + "best_epoch": 8, + "seconds": 93.71329402923584 + }, + { + "id": "formal_0108", + "tag": "representation_256", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8710626390184402, + "fixed25_actual_fraction": 0.2497668077778575, + "fixed25_rescue": 1179, + "fixed25_harm": 189, + "rank25_metric": 0.8711343904714071, + "validation_selection": 0.838384417072472, + "best_epoch": 5, + "seconds": 75.97537636756897 + }, + { + "id": "formal_0109", + "tag": "student_1M", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8709191361125063, + "fixed25_actual_fraction": 0.25400014350290595, + "fixed25_rescue": 1153, + "fixed25_harm": 165, + "rank25_metric": 0.8695558585061347, + "validation_selection": 0.8383414494414207, + "best_epoch": 7, + "seconds": 83.97415852546692 + }, + { + "id": "formal_0110", + "tag": "student_2M", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8709191361125063, + "fixed25_actual_fraction": 0.25400014350290595, + "fixed25_rescue": 1153, + "fixed25_harm": 165, + "rank25_metric": 0.8695558585061347, + "validation_selection": 0.8383414494414207, + "best_epoch": 7, + "seconds": 77.26504230499268 + }, + { + "id": "formal_0111", + "tag": "target_error", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8708473846595394, + "fixed25_actual_fraction": 0.24926454760708905, + "fixed25_rescue": 1183, + "fixed25_harm": 196, + "rank25_metric": 0.8709908875654732, + "validation_selection": 0.8376109997135491, + "best_epoch": 7, + "seconds": 75.30136489868164 + }, + { + "id": "formal_0112", + "tag": "target_four_state", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8713496448303079, + "fixed25_actual_fraction": 0.2545024036736744, + "fixed25_rescue": 1162, + "fixed25_harm": 168, + "rank25_metric": 0.8704168759417378, + "validation_selection": 0.8381839014608995, + "best_epoch": 4, + "seconds": 66.18311429023743 + }, + { + "id": "formal_0113", + "tag": "harm_cost_2", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8713496448303079, + "fixed25_actual_fraction": 0.2547176580325752, + "fixed25_rescue": 1160, + "fixed25_harm": 166, + "rank25_metric": 0.8700581186769032, + "validation_selection": 0.8382411916356345, + "best_epoch": 4, + "seconds": 65.56374073028564 + }, + { + "id": "formal_0209", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8323886058692689, + "fixed25_actual_fraction": 0.24072612470402527, + "fixed25_rescue": 1189, + "fixed25_harm": 239, + "rank25_metric": 0.8343258950993757, + "validation_selection": 0.7950730449727872, + "best_epoch": 4, + "seconds": 18.022375106811523 + }, + { + "id": "formal_0210", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8337518834756403, + "fixed25_actual_fraction": 0.24704025256511444, + "fixed25_rescue": 1213, + "fixed25_harm": 244, + "rank25_metric": 0.8344693980053096, + "validation_selection": 0.7959610426811802, + "best_epoch": 8, + "seconds": 38.606887102127075 + }, + { + "id": "extra_0037", + "tag": "representation_32", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 32, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8343258950993757, + "fixed25_actual_fraction": 0.24761426418884983, + "fixed25_rescue": 1213, + "fixed25_harm": 236, + "rank25_metric": 0.8349716581760781, + "validation_selection": 0.7954311085648811, + "best_epoch": 10, + "seconds": 90.23428177833557 + }, + { + "id": "extra_0038", + "tag": "representation_64", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 64, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8354021668938796, + "fixed25_actual_fraction": 0.24826002726555213, + "fixed25_rescue": 1229, + "fixed25_harm": 237, + "rank25_metric": 0.8360479299705819, + "validation_selection": 0.7952162704096247, + "best_epoch": 5, + "seconds": 69.96001935005188 + }, + { + "id": "extra_0039", + "tag": "student_4M", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8341823921934419, + "fixed25_actual_fraction": 0.24632273803544522, + "fixed25_rescue": 1212, + "fixed25_harm": 237, + "rank25_metric": 0.835043409629045, + "validation_selection": 0.7951876253222573, + "best_epoch": 3, + "seconds": 61.948747873306274 + }, + { + "id": "formal_0139", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8327473631341035, + "fixed25_actual_fraction": 0.24208940231039677, + "fixed25_rescue": 1196, + "fixed25_harm": 241, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7952162704096247, + "best_epoch": 3, + "seconds": 136.20692539215088 + }, + { + "id": "formal_0140", + "tag": "drop_image", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.834110640740475, + "fixed25_actual_fraction": 0.2450312118820406, + "fixed25_rescue": 1213, + "fixed25_harm": 239, + "rank25_metric": 0.8351151610820119, + "validation_selection": 0.7956459467201376, + "best_epoch": 3, + "seconds": 78.08365845680237 + }, + { + "id": "formal_0141", + "tag": "drop_text", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.832245102963335, + "fixed25_actual_fraction": 0.24201765085742985, + "fixed25_rescue": 1189, + "fixed25_harm": 241, + "rank25_metric": 0.8338236349286073, + "validation_selection": 0.7942280148954455, + "best_epoch": 3, + "seconds": 116.7338719367981 + }, + { + "id": "formal_0142", + "tag": "drop_hidden", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8328908660400373, + "fixed25_actual_fraction": 0.24194589940446293, + "fixed25_rescue": 1198, + "fixed25_harm": 241, + "rank25_metric": 0.8343258950993757, + "validation_selection": 0.7951589802348897, + "best_epoch": 2, + "seconds": 108.56272602081299 + }, + { + "id": "formal_0143", + "tag": "drop_confidence", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8183970725407189, + "fixed25_actual_fraction": 0.2401521130802899, + "fixed25_rescue": 968, + "fixed25_harm": 213, + "rank25_metric": 0.8196885986941236, + "validation_selection": 0.7879260956745918, + "best_epoch": 9, + "seconds": 167.05382919311523 + }, + { + "id": "formal_0144", + "tag": "hidden_position_ordered_image_text__layer_02", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_02", + "fixed25_metric": 0.833249623304872, + "fixed25_actual_fraction": 0.241874147951496, + "fixed25_rescue": 1199, + "fixed25_harm": 237, + "rank25_metric": 0.8349716581760781, + "validation_selection": 0.7944428530507018, + "best_epoch": 3, + "seconds": 115.09392309188843 + }, + { + "id": "formal_0145", + "tag": "hidden_position_ordered_image_text__layer_08", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_08", + "fixed25_metric": 0.834038889287508, + "fixed25_actual_fraction": 0.244672454617206, + "fixed25_rescue": 1215, + "fixed25_harm": 242, + "rank25_metric": 0.8351151610820119, + "validation_selection": 0.7938699513033515, + "best_epoch": 2, + "seconds": 116.08835816383362 + }, + { + "id": "formal_0146", + "tag": "hidden_multi_layer", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_02", + "fixed25_metric": 0.8327473631341035, + "fixed25_actual_fraction": 0.24208940231039677, + "fixed25_rescue": 1193, + "fixed25_harm": 238, + "rank25_metric": 0.834110640740475, + "validation_selection": 0.7944714981380693, + "best_epoch": 3, + "seconds": 122.4584412574768 + }, + { + "id": "formal_0147", + "tag": "representation_128", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 128, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8329626174930043, + "fixed25_actual_fraction": 0.2450312118820406, + "fixed25_rescue": 1194, + "fixed25_harm": 236, + "rank25_metric": 0.8342541436464088, + "validation_selection": 0.7950157547980522, + "best_epoch": 8, + "seconds": 162.51085686683655 + }, + { + "id": "formal_0148", + "tag": "representation_256", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 256, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8347564038171773, + "fixed25_actual_fraction": 0.25163234555499747, + "fixed25_rescue": 1224, + "fixed25_harm": 241, + "rank25_metric": 0.8344693980053096, + "validation_selection": 0.7955456889143512, + "best_epoch": 5, + "seconds": 153.2538034915924 + }, + { + "id": "formal_0149", + "tag": "student_1M", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8339671378345411, + "fixed25_actual_fraction": 0.24287866829303292, + "fixed25_rescue": 1208, + "fixed25_harm": 236, + "rank25_metric": 0.8352586639879458, + "validation_selection": 0.7945287883128044, + "best_epoch": 3, + "seconds": 137.92107677459717 + }, + { + "id": "formal_0150", + "tag": "student_2M", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8330343689459712, + "fixed25_actual_fraction": 0.24510296333500753, + "fixed25_rescue": 1202, + "fixed25_harm": 243, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7952878831280435, + "best_epoch": 3, + "seconds": 123.86622095108032 + }, + { + "id": "formal_0151", + "tag": "target_error", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8310253282628973, + "fixed25_actual_fraction": 0.24029561598622373, + "fixed25_rescue": 1177, + "fixed25_harm": 246, + "rank25_metric": 0.8339671378345411, + "validation_selection": 0.7920509882555142, + "best_epoch": 4, + "seconds": 133.07174134254456 + }, + { + "id": "formal_0152", + "tag": "target_four_state", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8327473631341035, + "fixed25_actual_fraction": 0.24208940231039677, + "fixed25_rescue": 1196, + "fixed25_harm": 241, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7952162704096247, + "best_epoch": 3, + "seconds": 134.1348536014557 + }, + { + "id": "formal_0153", + "tag": "fusion_concat", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8328908660400373, + "fixed25_actual_fraction": 0.24366793427566907, + "fixed25_rescue": 1196, + "fixed25_harm": 239, + "rank25_metric": 0.8348999067231111, + "validation_selection": 0.7947149813806932, + "best_epoch": 3, + "seconds": 95.99445247650146 + }, + { + "id": "formal_0166", + "tag": "target_error", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8317428427925665, + "fixed25_actual_fraction": 0.2414436392336945, + "fixed25_rescue": 1188, + "fixed25_harm": 247, + "rank25_metric": 0.8343258950993757, + "validation_selection": 0.7920223431681467, + "best_epoch": 3, + "seconds": 106.3988299369812 + }, + { + "id": "formal_0211", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8336083805697065, + "fixed25_actual_fraction": 0.24431369735237138, + "fixed25_rescue": 1234, + "fixed25_harm": 267, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7945717559438557, + "best_epoch": 2, + "seconds": 12.189603567123413 + }, + { + "id": "formal_0212", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8308818253569635, + "fixed25_actual_fraction": 0.2361340317141422, + "fixed25_rescue": 1178, + "fixed25_harm": 249, + "rank25_metric": 0.834110640740475, + "validation_selection": 0.7955027212832999, + "best_epoch": 3, + "seconds": 18.873005151748657 + }, + { + "id": "extra_0040", + "tag": "representation_32", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8328191145870704, + "fixed25_actual_fraction": 0.23677979479084452, + "fixed25_rescue": 1210, + "fixed25_harm": 254, + "rank25_metric": 0.8352586639879458, + "validation_selection": 0.7952735605843597, + "best_epoch": 2, + "seconds": 48.64926338195801 + }, + { + "id": "extra_0041", + "tag": "representation_64", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8327473631341035, + "fixed25_actual_fraction": 0.2382865753031499, + "fixed25_rescue": 1207, + "fixed25_harm": 252, + "rank25_metric": 0.8351151610820119, + "validation_selection": 0.794786594099112, + "best_epoch": 2, + "seconds": 49.1362726688385 + }, + { + "id": "extra_0042", + "tag": "student_4M", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8335366291167396, + "fixed25_actual_fraction": 0.24072612470402527, + "fixed25_rescue": 1222, + "fixed25_harm": 256, + "rank25_metric": 0.8348999067231111, + "validation_selection": 0.7951160126038385, + "best_epoch": 1, + "seconds": 45.84835124015808 + }, + { + "id": "formal_0151", + "tag": "target_error", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8307383224510296, + "fixed25_actual_fraction": 0.24029561598622373, + "fixed25_rescue": 1214, + "fixed25_harm": 287, + "rank25_metric": 0.8330343689459712, + "validation_selection": 0.7934116299054712, + "best_epoch": 4, + "seconds": 133.07174134254456 + }, + { + "id": "formal_0154", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8329626174930043, + "fixed25_actual_fraction": 0.24094137906292604, + "fixed25_rescue": 1225, + "fixed25_harm": 267, + "rank25_metric": 0.8348281552701442, + "validation_selection": 0.7948438842738471, + "best_epoch": 2, + "seconds": 78.18392181396484 + }, + { + "id": "formal_0155", + "tag": "drop_image", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8321733515103681, + "fixed25_actual_fraction": 0.23692329769677836, + "fixed25_rescue": 1202, + "fixed25_harm": 255, + "rank25_metric": 0.8348281552701442, + "validation_selection": 0.7950300773417358, + "best_epoch": 1, + "seconds": 70.36687850952148 + }, + { + "id": "formal_0156", + "tag": "drop_text", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8335366291167396, + "fixed25_actual_fraction": 0.24302217119896677, + "fixed25_rescue": 1222, + "fixed25_harm": 256, + "rank25_metric": 0.8354021668938796, + "validation_selection": 0.7942136923517618, + "best_epoch": 5, + "seconds": 98.69216632843018 + }, + { + "id": "formal_0157", + "tag": "drop_hidden", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8338953863815742, + "fixed25_actual_fraction": 0.2443854488053383, + "fixed25_rescue": 1243, + "fixed25_harm": 272, + "rank25_metric": 0.8348281552701442, + "validation_selection": 0.7966342022343168, + "best_epoch": 3, + "seconds": 74.8990888595581 + }, + { + "id": "formal_0158", + "tag": "drop_confidence", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8091411351079859, + "fixed25_actual_fraction": 0.2415871421396283, + "fixed25_rescue": 854, + "fixed25_harm": 228, + "rank25_metric": 0.8104326612613906, + "validation_selection": 0.7869378401604126, + "best_epoch": 7, + "seconds": 135.98869681358337 + }, + { + "id": "formal_0159", + "tag": "hidden_position_ordered_image_text__layer_02", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_02", + "fixed25_metric": 0.8321016000574012, + "fixed25_actual_fraction": 0.24194589940446293, + "fixed25_rescue": 1211, + "fixed25_harm": 265, + "rank25_metric": 0.8343258950993757, + "validation_selection": 0.7966055571469493, + "best_epoch": 3, + "seconds": 94.14239645004272 + }, + { + "id": "formal_0160", + "tag": "hidden_position_ordered_image_text__layer_08", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_08", + "fixed25_metric": 0.8323886058692689, + "fixed25_actual_fraction": 0.24122838487479373, + "fixed25_rescue": 1219, + "fixed25_harm": 269, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7952162704096248, + "best_epoch": 1, + "seconds": 75.29347610473633 + }, + { + "id": "formal_0161", + "tag": "hidden_multi_layer", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_02", + "fixed25_metric": 0.8319580971514673, + "fixed25_actual_fraction": 0.23993685872138912, + "fixed25_rescue": 1221, + "fixed25_harm": 277, + "rank25_metric": 0.8336083805697065, + "validation_selection": 0.7959610426811802, + "best_epoch": 1, + "seconds": 71.51151084899902 + }, + { + "id": "formal_0162", + "tag": "representation_128", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8326038602281696, + "fixed25_actual_fraction": 0.2398651072684222, + "fixed25_rescue": 1214, + "fixed25_harm": 261, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7945717559438557, + "best_epoch": 1, + "seconds": 71.7733223438263 + }, + { + "id": "formal_0163", + "tag": "representation_256", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8323168544163019, + "fixed25_actual_fraction": 0.24065437325105835, + "fixed25_rescue": 1209, + "fixed25_harm": 260, + "rank25_metric": 0.8342541436464088, + "validation_selection": 0.7948295617301633, + "best_epoch": 2, + "seconds": 87.87156558036804 + }, + { + "id": "formal_0164", + "tag": "student_1M", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8338236349286073, + "fixed25_actual_fraction": 0.24165889359259524, + "fixed25_rescue": 1226, + "fixed25_harm": 256, + "rank25_metric": 0.8364784386883835, + "validation_selection": 0.7949298195359495, + "best_epoch": 5, + "seconds": 109.21437549591064 + }, + { + "id": "formal_0165", + "tag": "student_8M", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8307383224510296, + "fixed25_actual_fraction": 0.23649278897897683, + "fixed25_rescue": 1181, + "fixed25_harm": 254, + "rank25_metric": 0.8338953863815742, + "validation_selection": 0.7947006588370095, + "best_epoch": 9, + "seconds": 120.51619029045105 + }, + { + "id": "formal_0166", + "tag": "target_error", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8300208079213605, + "fixed25_actual_fraction": 0.2402238645332568, + "fixed25_rescue": 1210, + "fixed25_harm": 293, + "rank25_metric": 0.8324603573222358, + "validation_selection": 0.7937124033228301, + "best_epoch": 4, + "seconds": 106.3988299369812 + }, + { + "id": "formal_0167", + "tag": "target_four_state", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8328908660400373, + "fixed25_actual_fraction": 0.23785606658534836, + "fixed25_rescue": 1190, + "fixed25_harm": 233, + "rank25_metric": 0.8354021668938796, + "validation_selection": 0.7952449154969923, + "best_epoch": 5, + "seconds": 106.75910758972168 + }, + { + "id": "formal_0168", + "tag": "fusion_attention", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8330343689459712, + "fixed25_actual_fraction": 0.23886058692688528, + "fixed25_rescue": 1206, + "fixed25_harm": 247, + "rank25_metric": 0.8355456697998135, + "validation_selection": 0.7952305929533086, + "best_epoch": 5, + "seconds": 144.87321782112122 + } + ] +} \ No newline at end of file diff --git a/paper_results/data/joint_MiniCPM_comparison_20260910.json b/paper_results/data/joint_MiniCPM_comparison_20260910.json new file mode 100644 index 0000000000000000000000000000000000000000..d6562c709e03367006f3212a57bfdc7387de3957 --- /dev/null +++ b/paper_results/data/joint_MiniCPM_comparison_20260910.json @@ -0,0 +1,58 @@ +[ + { + "expert": "cub", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.7119433897134967, + "multi_task_test_fixed25": 0.7079737659647911, + "delta_multi_minus_single": -0.003969623748705575 + }, + { + "expert": "glsim", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.759233690024163, + "multi_task_test_fixed25": 0.7745944080082844, + "delta_multi_minus_single": 0.01536071798412142 + }, + { + "expert": "nlvr2", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.8346846523642104, + "multi_task_test_fixed25": 0.8309535768099304, + "delta_multi_minus_single": -0.0037310755542799834 + }, + { + "expert": "vilt", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.8046925450240368, + "multi_task_test_fixed25": 0.802611752887996, + "delta_multi_minus_single": -0.002080792136040732 + }, + { + "expert": "groundingdino", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.3598105663159686, + "multi_task_test_fixed25": 0.37004792558772653, + "delta_multi_minus_single": 0.010237359271757951 + }, + { + "expert": "instancevg", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.6394237586138446, + "multi_task_test_fixed25": 0.6243654822335025, + "delta_multi_minus_single": -0.015058276380342073 + }, + { + "expert": "yolo26x", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.18032734231886507, + "multi_task_test_fixed25": 0.1836164473038217, + "delta_multi_minus_single": 0.003289104984956631 + }, + { + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.10367040308754236, + "multi_task_test_fixed25": 0.11260360812088349, + "delta_multi_minus_single": 0.008933205033341135 + } +] \ No newline at end of file diff --git a/paper_results/data/joint_Qwen3.8_comparison_20260910.json b/paper_results/data/joint_Qwen3.8_comparison_20260910.json new file mode 100644 index 0000000000000000000000000000000000000000..e5487d71fdc8487a4580f45e2ae7a63bffdc98f4 --- /dev/null +++ b/paper_results/data/joint_Qwen3.8_comparison_20260910.json @@ -0,0 +1,58 @@ +[ + { + "expert": "cub", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.7619951674145667, + "multi_task_test_fixed25": 0.7782188470831896, + "delta_multi_minus_single": 0.016223679668622815 + }, + { + "expert": "glsim", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.8301691404901622, + "multi_task_test_fixed25": 0.8448394891266828, + "delta_multi_minus_single": 0.014670348636520614 + }, + { + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.8372677046710196, + "multi_task_test_fixed25": 0.8337518834756403, + "delta_multi_minus_single": -0.0035158211953792984 + }, + { + "expert": "vilt", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.8045490421181029, + "multi_task_test_fixed25": 0.8067733371600775, + "delta_multi_minus_single": 0.0022242950419746332 + }, + { + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.4223974137197629, + "multi_task_test_fixed25": 0.45991549215892014, + "delta_multi_minus_single": 0.03751807843915722 + }, + { + "expert": "instancevg", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.7224853245611548, + "multi_task_test_fixed25": 0.7053001729858492, + "delta_multi_minus_single": -0.017185151575305535 + }, + { + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.19009376433100783, + "multi_task_test_fixed25": 0.19603467084956394, + "delta_multi_minus_single": 0.0059409065185561105 + }, + { + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.10483176081963008, + "multi_task_test_fixed25": 0.12038495915476813, + "delta_multi_minus_single": 0.015553198335138047 + } +] \ No newline at end of file diff --git a/paper_results/data/layout_verification.json b/paper_results/data/layout_verification.json new file mode 100644 index 0000000000000000000000000000000000000000..5fcd538e4ee47c5ead19602b1a6b9f6a27ca963b --- /dev/null +++ b/paper_results/data/layout_verification.json @@ -0,0 +1,29 @@ +{ + "old_distribution": { + "width_pt": 523.1995239257812, + "height_pt": 309.7997741699219, + "height_at_7_16_in": 4.239618504261674 + }, + "old_budget": { + "width_pt": 526.7994995117188, + "height_pt": 237.7997589111328, + "height_at_7_16_in": 3.2320575007794505 + }, + "combined": { + "width_pt": 515.52001953125, + "height_pt": 324.0, + "height_at_7_16_in": 4.499999829510743 + }, + "reduction_percent": 39.77255134624947, + "old_pages": 16, + "new_pages": 14, + "new_figure_and_section_pages": [ + 6, + 7 + ], + "changed_sections": [ + "sections\\appendix.tex", + "sections\\experiments.tex" + ], + "figure1_identical": true +} \ No newline at end of file diff --git a/paper_results/data/main_table_cost_qwen_20260912/build_figures.py b/paper_results/data/main_table_cost_qwen_20260912/build_figures.py new file mode 100644 index 0000000000000000000000000000000000000000..5953cf95b5fd0da7cd8d67f73fce7c796cae7941 --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/build_figures.py @@ -0,0 +1,101 @@ +"""Main-table curves plus fresh measured component costs; no threshold re-selection.""" +from pathlib import Path +import csv +import json +import numpy as np +import matplotlib +matplotlib.use('Agg') +import matplotlib.pyplot as plt +from matplotlib.ticker import MaxNLocator,LogLocator,NullLocator + +root=Path(__file__).resolve().parent +groups=json.loads((root/'main_table_results.json').read_text(encoding='utf-8'))['groups'] +components={} +curves=[] +selected=[] +for g,tag in zip(groups,['qwen_gref','qwen']): + expert=g['key'][1] + spec=json.loads((root/f'components/{expert}_specialist.json').read_text()) + router=json.loads((root/f'components/{expert}_router.json').read_text()) + large=json.loads((root/f'components/{tag}_latency.json').read_text()) + flops=json.loads((root/f'components/{tag}_flops.json').read_text()) + assert spec['completed'] and router['status']==large['status']==flops['status']=='complete' + assert router['run']==g['run'] and router['selected_decisions_match_main'] + assert abs(router['actual_call_fraction']-g['methods']['learned']['test_point']['actual_fraction'])<1e-12 + key='|'.join(g['key']) + c={'specialist_gflops':spec['mean_counted_gflops_per_sample'], + 'router_gflops':router['mean_gflops_per_input'], + 'large_gflops':flops['mean_prefill_gflops_counted']+large['mean_completion_tokens']*flops['mean_decode_one_token_gflops_counted'], + 'specialist_ms':spec['mean_end_to_end_latency_ms_per_sample'], + 'router_ms':router['mean_latency_ms_per_input'], + 'large_ms':large['mean_batch_wall_ms_per_sample'], + 'large_samples':large['measured_count'], 'flops_probes':flops['sample_count'], + 'specialist_samples':spec['sample_count'],'router_samples':router['sample_count'],'batch_size':large['batch_size']} + components[key]=c + for method in ['learned','confidence']: + for kind,points in [('curve',g['methods'][method]['test_curve']),('selected',[g['methods'][method]['test_point']])]: + for p in points: + rate=p['actual_fraction'] + row={'pair':key,'method':method,'nominal_validation_budget':p['budget'],'actual_test_call_fraction':rate, + 'test_metric':p['metric'],'estimated_gflops_per_input':c['specialist_gflops']+(c['router_gflops'] if method=='learned' else 0)+rate*c['large_gflops'], + 'estimated_time_ms_per_input':c['specialist_ms']+(c['router_ms'] if method=='learned' else 0)+rate*c['large_ms']} + (curves if kind=='curve' else selected).append(row) +for name,rows in [('full_cost_performance_curves',curves),('selected_points',selected),('fixed_budget_cost_performance',[p for p in curves if p['nominal_validation_budget'] in [0,.05,.1,.25,.5,.75,1]])]: + with (root/(name+'.csv')).open('w',encoding='utf-8',newline='') as f: + writer=csv.DictWriter(f,fieldnames=list(rows[0]));writer.writeheader();writer.writerows(rows) +summary={'components':components,'selected_points':selected,'protocol': + {'method':'original main-table logit contrast; original checkpoints and validation thresholds; seed 42', + 'large_timing':'64 measured examples; batch size one; same full task prompt; two separate warmup examples', + 'cost':'serial component estimates, not a timed integrated service; includes active distilled encoders and routing head only for learned', + 'flops':'multiply-add=2; supported operators; mean prefill (including visual embedding) plus average generated length times one-token decode probe', + 'scope_limits':'mean call cost extrapolated across policies; no policy-specific input/output-length costs; specialist-feature extraction overhead is not separately measured; confidence threshold overhead not separately timed'}} +(root/'cost_summary.json').write_text(json.dumps(summary,indent=2),encoding='utf-8') + +plt.rcParams.update({'font.family':'serif','font.serif':['Times New Roman'],'font.size':7, + 'axes.labelsize':7,'xtick.labelsize':6.5,'ytick.labelsize':6.5,'legend.fontsize':7, + 'pdf.fonttype':42,'ps.fonttype':42,'axes.spines.top':False,'axes.spines.right':False, + 'axes.linewidth':.55,'mathtext.fontset':'stix'}) +styles=[('learned','learned','#176f91','-'),('confidence','conf.','#c78324','--')] +keys=list(components) + +def panel(ax,key,field): + for method,label,color,ls in styles: + pts=[p for p in curves if p['pair']==key and p['method']==method] + assert len(pts)==101 + x=np.array([p[field] for p in pts]);y=np.array([p['test_metric']*100 for p in pts]) + assert np.all(np.diff(x)>=-1e-7) + ax.plot(x,y,color=color,ls=ls,lw=1.05,label=label) + p=next(p for p in selected if p['pair']==key and p['method']==method) + ax.scatter(p[field],100*p['test_metric'],color=color,s=13,edgecolor='white',linewidth=.4,zorder=5) + ax.grid(axis='y',color='#dce1e4',lw=.4) + ax.tick_params(length=2.3,pad=1.5) + ax.yaxis.set_major_locator(MaxNLocator(nbins=3)) + if 'gflops' in field: + ax.set_xscale('log');ax.xaxis.set_major_locator(LogLocator(base=10,numticks=4));ax.xaxis.set_minor_locator(NullLocator()) + ax.set_xlabel('Est. GFLOPs/input',labelpad=1) + else: + ax.xaxis.set_major_locator(MaxNLocator(nbins=3));ax.set_xlabel('Est. time (ms/input)',labelpad=1) + ax.set_ylabel('Acc. (%)' if key.startswith('grefcoco') else 'Macro-F1 (%)',labelpad=1) + +for field,stem in [('estimated_gflops_per_input','flops_performance'),('estimated_time_ms_per_input','latency_performance')]: + fig,axes=plt.subplots(1,2,figsize=(3.44,1.50)) + fig.subplots_adjust(left=.115,right=.995,bottom=.31,top=.83,wspace=.43) + for ax,key,title in zip(axes,keys,['gRefCOCO / Q','Construction / Q']): + panel(ax,key,field) + ax.text(.5,1.10,title,transform=ax.transAxes,ha='center',fontsize=7) + h,l=axes[0].get_legend_handles_labels() + fig.legend(h,l,loc='lower center',bbox_to_anchor=(.53,-.025),ncol=2,frameon=False,handlelength=2.2,columnspacing=1.8) + fig.savefig(root/(stem+'.pdf'));fig.savefig(root/(stem+'.png'),dpi=400) + plt.close(fig) + +fig,axes=plt.subplots(2,2,figsize=(3.44,2.05)) +fig.subplots_adjust(left=.115,right=.995,bottom=.20,top=.89,wspace=.43,hspace=.64) +for col,key in enumerate(keys): + for row,field in enumerate(['estimated_gflops_per_input','estimated_time_ms_per_input']): + panel(axes[row,col],key,field) + axes[0,col].text(.5,1.14,'gRefCOCO / Q' if col==0 else 'Construction / Q',transform=axes[0,col].transAxes,ha='center',fontsize=7) +h,l=axes[0,0].get_legend_handles_labels() +fig.legend(h,l,loc='lower center',bbox_to_anchor=(.53,-.005),ncol=2,frameon=False,handlelength=2.2,columnspacing=1.8) +fig.savefig(root/'cost_compact.pdf');fig.savefig(root/'cost_compact.png',dpi=400) +plt.close(fig) +print(json.dumps(summary,indent=2)) diff --git a/paper_results/data/main_table_cost_qwen_20260912/components/instancevg_router.json b/paper_results/data/main_table_cost_qwen_20260912/components/instancevg_router.json new file mode 100644 index 0000000000000000000000000000000000000000..193120495a2aa267d7efb03a46542590614d2e9e --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/components/instancevg_router.json @@ -0,0 +1,239 @@ +{ + "status": "complete", + "run": "shared_12_instancevg_Qwen3.8", + "config": { + "fraction": 1.0, + "seed": 42, + "freeze_encoder": true, + "fusion": "concat", + "harm_cost": 1.0, + "batch_size": 256, + "weight_decay": 0.01, + "max_epochs": 50, + "patience": 8, + "cpu_threads": 1, + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90, + "expert": "instancevg", + "task": "grefcoco", + "endpoint": "Qwen3.8", + "branches": [ + "image", + "text", + "hidden", + "confidence", + "output" + ], + "hidden_layer": "joint_encoder__layer_11", + "id": "shared_12_instancevg_Qwen3.8", + "stage": "shared_mandatory_inputs" + }, + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_shared_image_hidden_20260910/runs/shared_12_instancevg_Qwen3.8/best.pt", + "score": "logit_rescue-logit_harm", + "test_rows_recomputed": 35263, + "selected_decisions_match_main": true, + "max_score_abs_difference": 0.0, + "actual_call_fraction": 0.5355755324277571, + "sample_count": 64, + "sample_ids": [ + "testB-5056-14012", + "testB-31505-86708", + "testA-57489-158264", + "testA-38865-107108", + "testB-47361-130361", + "testA-5476-15123", + "testA-71404-196892", + "testA-61030-168115", + "testB-35969-99063", + "testB-1675-4724", + "testA-30414-83641", + "testB-47742-131425", + "testB-77456-213280", + "testA-59283-163253", + "testA-57390-157996", + "testA-77229-212665", + "testA-6784-18703", + "testB-11434-31542", + "testB-11675-32181", + "testB-73046-201304", + "testA-28837-79356", + "testA-4563-12619", + "testB-30168-82996", + "testB-92461-254721", + "testA-58293-160462", + "testA-13050-35910", + "testA-39320-108309", + "testB-36423-100313", + "testA-43418-119528", + "testB-23411-64351", + "testB-77730-214031", + "testA-7232-19907", + "testA-17738-48695", + "testA-44914-123593", + "testB-92861-255854", + "testA-69678-192132", + "testA-68785-189658", + "testB-62911-173380", + "testB-19029-52242", + "testA-66284-182759", + "testB-19170-52626", + "testB-8468-23291", + "testB-72940-201009", + "testB-74045-203983", + "testB-93380-257275", + "testB-49369-135990", + "testA-58046-159759", + "testA-57437-158132", + "testA-5754-15895", + "testA-85590-235577", + "testA-92278-254227", + "testB-80373-221257", + "testA-80383-221294", + "testB-71197-196311", + "testB-48080-132352", + "testB-18351-50380", + "testB-1089-3077", + "testB-92466-254737", + "testB-53065-146099", + "testB-42611-117349", + "testA-57690-158796", + "testA-65173-179681", + "testB-90675-249707", + "testB-74709-205815" + ], + "batch_size": 1, + "latency_ms": [ + 10.095497942529619, + 10.442811995744705, + 9.535272954963148, + 8.56779795140028, + 28.526668902486563, + 14.110558084212244, + 19.977235002443194, + 21.489594015292823, + 16.842726967297494, + 12.316308100707829, + 12.318658991716802, + 13.136874069459736, + 16.18148700799793, + 11.749238008633256, + 12.584974057972431, + 12.680649990215898, + 11.369985993951559, + 14.251800021156669, + 14.000480994582176, + 12.42606493178755, + 13.526970986276865, + 11.346371960826218, + 11.965820915065706, + 13.418521964922547, + 13.269996969029307, + 13.868742040358484, + 11.83428696822375, + 13.831221032887697, + 12.576585984788835, + 12.408879934810102, + 12.892312020994723, + 15.224601957015693, + 13.460875954478979, + 11.89423305913806, + 10.926949908025563, + 157.88777999114245, + 16.57543599139899, + 29.21812597196549, + 23.506848025135696, + 13.830928015522659, + 14.57549398764968, + 13.596423901617527, + 16.056897002272308, + 12.610359000973403, + 17.024922999553382, + 12.336026062257588, + 14.77054797578603, + 13.220850960351527, + 19.9308970477432, + 16.4189770584926, + 17.153396038338542, + 31.296705012209713, + 20.386314019560814, + 14.672313001938164, + 15.437994967214763, + 10.980968945659697, + 15.224996954202652, + 12.009209021925926, + 12.397779966704547, + 13.347716070711613, + 15.58609597850591, + 17.80865201726556, + 13.338097953237593, + 13.845606008544564 + ], + "mean_latency_ms_per_input": 17.032772149832454, + "head_mean_latency_ms": 0.6086819972842932, + "gflops_per_input": [ + 3.642171392, + 3.642171392, + 3.642171392, + 3.642171392, + 3.642171392, + 3.642171392, + 3.642171392, + 3.642171392 + ], + "mean_gflops_per_input": 3.642171392, + "operators": [ + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + } + ], + "scope": "image loading, bilinear resize, tokenize when used, two image slots (including gray placeholder), active distilled encoders, feature assembly and original routing head; cached specialist features; supported FLOPs only" +} \ No newline at end of file diff --git a/paper_results/data/main_table_cost_qwen_20260912/components/instancevg_specialist.json b/paper_results/data/main_table_cost_qwen_20260912/components/instancevg_specialist.json new file mode 100644 index 0000000000000000000000000000000000000000..aebe35a841d23d1c4030450bea8661304d1e417f --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/components/instancevg_specialist.json @@ -0,0 +1,254 @@ +{ + "component": "specialist", + "expert": "instancevg", + "task": "grefcoco", + "device": "cuda:0", + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/models/grefcoco/instancevg/InstanceVG-grefcoco.pth", + "sample_count": 64, + "batch_size": 1, + "sample_ids": [ + "testA-30844-84861", + "testB-44353-122077", + "testB-36889-101643", + "testA-49055-135087", + "testA-81245-223633", + "testB-37669-103833", + "testA-5038-13943", + "testB-79646-219274", + "testA-469-1309", + "testB-46644-128438", + "testA-24562-67515", + "testA-47070-129599", + "testA-54554-150141", + "testB-12382-34099", + "testB-23014-63279", + "testB-79647-219275", + "testB-69479-191564", + "testA-15582-42797", + "testB-56692-156000", + "testB-42726-117660", + "testA-80455-221494", + "testB-33115-91174", + "testA-62940-173455", + "testB-31494-86670", + "testA-54586-150233", + "testA-81445-224177", + "testB-8417-23141", + "testA-24438-67190", + "testA-59285-163255", + "testA-73258-201893", + "testB-945-2633", + "testA-54307-149426", + "testB-60557-166822", + "testA-81622-224665", + "testA-69466-191525", + "testA-60731-167321", + "testA-7857-21644", + "testA-44142-121499", + "testB-66162-182407", + "testA-31764-87406", + "testB-38637-106482", + "testB-52921-145709", + "testB-65796-181394", + "testB-2936-8113", + "testA-79438-218690", + "testA-57397-158016", + "testB-23920-65743", + "testB-15419-42358", + "testA-10078-27739", + "testA-43890-120793", + "testA-55116-151689", + "testB-27487-75582", + "testA-26856-73830", + "testB-50845-140107", + "testA-29199-80374", + "testB-65935-181778", + "testA-17736-48684", + "testB-69477-191553", + "testB-22290-61227", + "testA-58476-160950", + "testB-36265-99859", + "testA-5734-15837", + "testB-67952-187353", + "testB-47054-129555" + ], + "model_load_seconds_excluded_from_latency": true, + "forward_latency_ms_per_sample": [ + 41.24195098876953, + 41.74086380004883, + 40.36540985107422, + 44.46102523803711, + 41.2421760559082, + 40.76591873168945, + 41.14704132080078, + 40.88172912597656, + 41.70716857910156, + 41.191871643066406, + 39.91231918334961, + 39.29497528076172, + 40.70723342895508, + 38.920894622802734, + 39.32780838012695, + 38.89958572387695, + 38.59721755981445, + 38.47331237792969, + 39.8744010925293, + 39.87260818481445, + 41.48284912109375, + 40.55779266357422, + 39.05420684814453, + 39.052734375, + 39.143775939941406, + 39.82249450683594, + 39.543006896972656, + 39.368160247802734, + 40.661216735839844, + 39.092193603515625, + 39.85852813720703, + 40.33164978027344, + 38.89593505859375, + 39.29216003417969, + 39.20943832397461, + 40.946590423583984, + 39.44643020629883, + 40.220863342285156, + 39.61030578613281, + 39.639007568359375, + 39.78163146972656, + 39.223201751708984, + 39.00921630859375, + 38.99894332885742, + 41.13721466064453, + 39.94972610473633, + 41.41766357421875, + 39.28073501586914, + 39.721214294433594, + 39.24937438964844, + 39.40460968017578, + 41.30179214477539, + 42.94054412841797, + 41.5063362121582, + 44.03267288208008, + 44.55561447143555, + 44.24995040893555, + 44.87868881225586, + 45.14652633666992, + 44.85055923461914, + 42.39228820800781, + 41.121856689453125, + 45.708351135253906, + 41.8752326965332 + ], + "end_to_end_latency_ms_per_sample": [ + 47.29630902875215, + 46.27182998228818, + 47.56885801907629, + 45.48592597711831, + 45.399176073260605, + 46.442538034170866, + 45.95420998521149, + 46.45739798434079, + 44.90085388533771, + 43.80481690168381, + 44.848884106613696, + 43.67590101901442, + 46.40063806436956, + 44.26892101764679, + 44.44277100265026, + 44.71806308720261, + 44.62323593907058, + 44.6264409692958, + 44.96095096692443, + 44.103492982685566, + 45.13338196557015, + 44.1846780013293, + 48.1606520479545, + 46.43543600104749, + 46.19023704435676, + 45.247678994201124, + 44.801001087762415, + 45.467636082321405, + 44.22409099061042, + 44.7448689956218, + 58.0457029864192, + 46.10048606991768, + 44.56539999227971, + 44.47658592835069, + 46.05250502936542, + 44.743441976606846, + 44.77924096863717, + 43.877715012058616, + 44.05005706939846, + 42.87725198082626, + 43.836933095008135, + 45.634721987880766, + 44.748114072717726, + 46.21323198080063, + 52.24333005025983, + 48.90036000870168, + 48.12136595137417, + 47.65231907367706, + 46.71858798246831, + 48.516221111640334, + 43.90157596208155, + 46.83465394191444, + 46.49809293914586, + 42.6300170365721, + 47.39421408157796, + 48.35387098137289, + 48.176085110753775, + 48.29539393540472, + 47.32040502130985, + 46.35261802468449, + 47.482443042099476, + 47.437409055419266, + 47.27676196489483, + 48.198972013778985 + ], + "mean_forward_latency_ms_per_sample": 40.71232491731644, + "median_forward_latency_ms_per_sample": 40.08529472351074, + "mean_end_to_end_latency_ms_per_sample": 46.08042127620138, + "median_end_to_end_latency_ms_per_sample": 45.79446598654613, + "flop_profiles": [ + { + "batch_size": 1, + "flops_counted": 7502680064, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 7500632064, + "aten.baddbmm": 2048000 + } + }, + { + "batch_size": 1, + "flops_counted": 7502680064, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 7500632064, + "aten.baddbmm": 2048000 + } + }, + { + "batch_size": 1, + "flops_counted": 7502680064, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 7500632064, + "aten.baddbmm": 2048000 + } + }, + { + "batch_size": 1, + "flops_counted": 7502680064, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 7500632064, + "aten.baddbmm": 2048000 + } + } + ], + "mean_counted_gflops_per_sample": 7.502680064, + "flops_scope": "GPU FlopCounterMode dense products, convolutions and supported attention; multiply-add counted as 2 FLOPs. Elementwise, normalization, custom ops and postprocessing may be missing.", + "latency_scope": "A800 wall-clock CUDA forward latency and end-to-end latency including image read/preprocess, excluding model load; batch processing uses the recorded batch size.", + "completed": true +} \ No newline at end of file diff --git a/paper_results/data/main_table_cost_qwen_20260912/components/qwen_flops.json b/paper_results/data/main_table_cost_qwen_20260912/components/qwen_flops.json new file mode 100644 index 0000000000000000000000000000000000000000..58b8c3f787acbc76f88e2d4185de8db41426bb45 --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/components/qwen_flops.json @@ -0,0 +1,341 @@ +{ + "status": "complete", + "model_kind": "Qwen3.8", + "task": "construction", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/Qwen3.8-27B-FP8", + "sample_count": 8, + "records": [ + { + "sample_id": "0000184", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0001955", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0000840", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0003887", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0003433", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0001613", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0002591", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0003105", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + } + ], + "mean_prefill_gflops_counted": 90083.398713024, + "mean_decode_one_token_gflops_counted": 51.922600128, + "scope": "HF reference execution on the A800 with real task inputs; dense products/convolutions/supported attention counted with multiply-add=2 FLOPs. vLLM custom kernels, elementwise/normalization/nonlinearities and other unsupported operations are not complete.", + "answer_generation": "not performed; one known-token decode probe only" +} \ No newline at end of file diff --git a/paper_results/data/main_table_cost_qwen_20260912/components/qwen_gref_flops.json b/paper_results/data/main_table_cost_qwen_20260912/components/qwen_gref_flops.json new file mode 100644 index 0000000000000000000000000000000000000000..ec5d0f5375ca215c84bf4c0eb314781a873d6ea1 --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/components/qwen_gref_flops.json @@ -0,0 +1,341 @@ +{ + "status": "complete", + "model_kind": "Qwen3.8", + "task": "grefcoco", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/Qwen3.8-27B-FP8", + "sample_count": 8, + "records": [ + { + "sample_id": "testA-5038-13945", + "prompt_tokens_in_reference_tensor": 608, + "input_shapes": { + "input_ids": [ + 1, + 608 + ], + "attention_mask": [ + 1, + 608 + ], + "mm_token_type_ids": [ + 1, + 608 + ], + "pixel_values": [ + 1200, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 31223201515520, + "decode_one_token_flops_counted": 51491242176, + "prefill_gflops_counted": 31223.20151552, + "decode_one_token_gflops_counted": 51.491242176, + "prefill_operators": { + "aten.convolution": 6649282560, + "aten.addmm": 1013465088000, + "aten.bmm": 590268057600, + "aten.mm": 29612819087360 + }, + "decode_operators": { + "aten.bmm": 239468736, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "testA-73305-202014", + "prompt_tokens_in_reference_tensor": 608, + "input_shapes": { + "input_ids": [ + 1, + 608 + ], + "attention_mask": [ + 1, + 608 + ], + "mm_token_type_ids": [ + 1, + 608 + ], + "pixel_values": [ + 1200, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 31223201515520, + "decode_one_token_flops_counted": 51491242176, + "prefill_gflops_counted": 31223.20151552, + "decode_one_token_gflops_counted": 51.491242176, + "prefill_operators": { + "aten.convolution": 6649282560, + "aten.addmm": 1013465088000, + "aten.bmm": 590268057600, + "aten.mm": 29612819087360 + }, + "decode_operators": { + "aten.bmm": 239468736, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "testA-26952-74099", + "prompt_tokens_in_reference_tensor": 562, + "input_shapes": { + "input_ids": [ + 1, + 562 + ], + "attention_mask": [ + 1, + 562 + ], + "mm_token_type_ids": [ + 1, + 562 + ], + "pixel_values": [ + 1040, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 28754745861504, + "decode_one_token_flops_counted": 51473154240, + "prefill_gflops_counted": 28754.745861504, + "decode_one_token_gflops_counted": 51.47315424, + "prefill_operators": { + "aten.convolution": 5902172160, + "aten.addmm": 878336409600, + "aten.bmm": 497939359104, + "aten.mm": 27372567920640 + }, + "decode_operators": { + "aten.bmm": 221380800, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "testB-53254-146620", + "prompt_tokens_in_reference_tensor": 560, + "input_shapes": { + "input_ids": [ + 1, + 560 + ], + "attention_mask": [ + 1, + 560 + ], + "mm_token_type_ids": [ + 1, + 560 + ], + "pixel_values": [ + 1040, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 28656453395456, + "decode_one_token_flops_counted": 51472367808, + "prefill_gflops_counted": 28656.453395456, + "decode_one_token_gflops_counted": 51.472367808, + "prefill_operators": { + "aten.convolution": 5894307840, + "aten.addmm": 878336409600, + "aten.bmm": 497056982016, + "aten.mm": 27275165696000 + }, + "decode_operators": { + "aten.bmm": 220594368, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "testB-37889-104439", + "prompt_tokens_in_reference_tensor": 687, + "input_shapes": { + "input_ids": [ + 1, + 687 + ], + "attention_mask": [ + 1, + 687 + ], + "mm_token_type_ids": [ + 1, + 687 + ], + "pixel_values": [ + 1520, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 35517384622912, + "decode_one_token_flops_counted": 51522306240, + "prefill_gflops_counted": 35517.384622912, + "decode_one_token_gflops_counted": 51.52230624, + "prefill_operators": { + "aten.convolution": 8092385280, + "aten.addmm": 1283722444800, + "aten.bmm": 765362832192, + "aten.mm": 33460206960640 + }, + "decode_operators": { + "aten.bmm": 270532800, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "testA-58523-161084", + "prompt_tokens_in_reference_tensor": 500, + "input_shapes": { + "input_ids": [ + 1, + 500 + ], + "attention_mask": [ + 1, + 500 + ], + "mm_token_type_ids": [ + 1, + 500 + ], + "pixel_values": [ + 768, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 25390700918528, + "decode_one_token_flops_counted": 51448774848, + "prefill_gflops_counted": 25390.700918528, + "decode_one_token_gflops_counted": 51.448774848, + "prefill_operators": { + "aten.convolution": 4695785472, + "aten.addmm": 648617656320, + "aten.bmm": 384288519936, + "aten.mm": 24353098956800 + }, + "decode_operators": { + "aten.bmm": 197001408, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "testB-3419-9449", + "prompt_tokens_in_reference_tensor": 568, + "input_shapes": { + "input_ids": [ + 1, + 568 + ], + "attention_mask": [ + 1, + 568 + ], + "mm_token_type_ids": [ + 1, + 568 + ], + "pixel_values": [ + 1040, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 29049642134016, + "decode_one_token_flops_counted": 51475513536, + "prefill_gflops_counted": 29049.642134016, + "decode_one_token_gflops_counted": 51.475513536, + "prefill_operators": { + "aten.convolution": 5925765120, + "aten.addmm": 878336409600, + "aten.bmm": 500605364736, + "aten.mm": 27664774594560 + }, + "decode_operators": { + "aten.bmm": 223740096, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "testB-24035-66084", + "prompt_tokens_in_reference_tensor": 561, + "input_shapes": { + "input_ids": [ + 1, + 561 + ], + "attention_mask": [ + 1, + 561 + ], + "mm_token_type_ids": [ + 1, + 561 + ], + "pixel_values": [ + 1040, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 28705599235264, + "decode_one_token_flops_counted": 51472761024, + "prefill_gflops_counted": 28705.599235264, + "decode_one_token_gflops_counted": 51.472761024, + "prefill_operators": { + "aten.convolution": 5898240000, + "aten.addmm": 878336409600, + "aten.bmm": 497497777344, + "aten.mm": 27323866808320 + }, + "decode_operators": { + "aten.bmm": 220987584, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + } + ], + "mean_prefill_gflops_counted": 29815.11614984, + "mean_decode_one_token_gflops_counted": 51.480920256000005, + "scope": "HF reference execution on the A800 with real task inputs; dense products/convolutions/supported attention counted with multiply-add=2 FLOPs. vLLM custom kernels, elementwise/normalization/nonlinearities and other unsupported operations are not complete.", + "answer_generation": "not performed; one known-token decode probe only" +} \ No newline at end of file diff --git a/paper_results/data/main_table_cost_qwen_20260912/components/qwen_gref_latency.json b/paper_results/data/main_table_cost_qwen_20260912/components/qwen_gref_latency.json new file mode 100644 index 0000000000000000000000000000000000000000..d23d091ce482278c9b6b8d326d0f9d0663f7c403 --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/components/qwen_gref_latency.json @@ -0,0 +1,2446 @@ +{ + "status": "complete", + "model_kind": "Qwen3.8", + "task": "grefcoco", + "expert": "instancevg", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/Qwen3.8-27B-FP8", + "device": "cuda:0", + "gpu_memory_utilization": 0.85, + "sample_count_requested": 66, + "warmup_count": 2, + "measured_count": 64, + "batch_size": 1, + "sample_ids": [ + "testA-29199-80373", + "testB-44351-122075", + "testA-91850-253019", + "testB-36888-101641", + "testA-54586-150233", + "testB-79644-219272", + "testB-65935-181776", + "testA-81622-224664", + "testA-10078-27739", + "testA-15582-42797", + "testA-5038-13943", + "testA-30844-84861", + "testB-15419-42356", + "testB-27486-75580", + "testB-945-2632", + "testA-24438-67190", + "testB-66161-182405", + "testB-42726-117659", + "testB-69439-191450", + "testA-54307-149426", + "testB-37669-103831", + "testB-50844-140105", + "testA-79438-218690", + "testA-62940-173454", + "testA-81245-223632", + "testA-47070-129599", + "testB-12381-34098", + "testA-44142-121498", + "testA-73258-201893", + "testB-46644-128436", + "testA-43889-120792", + "testB-36082-99363", + "testB-33108-91152", + "testA-26856-73830", + "testA-81271-223709", + "testA-54554-150141", + "testB-38636-106481", + "testA-5734-15837", + "testB-67558-186325", + "testB-47053-129554", + "testA-93064-256422", + "testB-60557-166821", + "testA-55116-151688", + "testB-65796-181392", + "testB-23014-63278", + "testA-24562-67515", + "testA-17736-48684", + "testA-58475-160949", + "testA-80455-221493", + "testA-31763-87405", + "testB-23920-65742", + "testA-469-1309", + "testB-56691-155998", + "testB-31493-86668", + "testA-59284-163254", + "testA-57397-158015", + "testB-2935-8111", + "testB-22290-61226", + "testB-8416-23140", + "testA-60731-167320", + "testA-49055-135086", + "testA-7857-21644", + "testB-79645-219273", + "testB-52921-145707", + "testB-69479-191562", + "testA-69466-191524" + ], + "batches": [ + { + "sample_ids": [ + "testA-91850-253019" + ], + "batch_size": 1, + "generation_latency_seconds": 3.147620694944635, + "batch_wall_latency_seconds": 3.1645371050108224, + "amortized_generation_ms_per_sample": 3147.620694944635, + "amortized_batch_wall_ms_per_sample": 3164.5371050108224, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [496, 292, 713, 655], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 496, + 292, + 713, + 655 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 39, + "prompt_tokens": 570 + } + ] + }, + { + "sample_ids": [ + "testB-36888-101641" + ], + "batch_size": 1, + "generation_latency_seconds": 4.945294587058015, + "batch_wall_latency_seconds": 4.966181440046057, + "amortized_generation_ms_per_sample": 4945.294587058015, + "amortized_batch_wall_ms_per_sample": 4966.181440046057, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[21,567,693,915],\"score\":0.95},{\"bbox_2d\":[381,233,959,885],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 21, + 567, + 693, + 915 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 381, + 233, + 959, + 885 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 60, + "prompt_tokens": 608 + } + ] + }, + { + "sample_ids": [ + "testA-54586-150233" + ], + "batch_size": 1, + "generation_latency_seconds": 3.2111774859949946, + "batch_wall_latency_seconds": 3.2314598000375554, + "amortized_generation_ms_per_sample": 3211.1774859949946, + "amortized_batch_wall_ms_per_sample": 3231.4598000375554, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [693, 438, 983, 813], \"score\": 0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 693, + 438, + 983, + 813 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 40, + "prompt_tokens": 707 + } + ] + }, + { + "sample_ids": [ + "testB-79644-219272" + ], + "batch_size": 1, + "generation_latency_seconds": 33.390286731068045, + "batch_wall_latency_seconds": 33.40712711191736, + "amortized_generation_ms_per_sample": 33390.286731068045, + "amortized_batch_wall_ms_per_sample": 33407.12711191736, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [121, 581, 170, 764], \"score\": 0.9}, {\"bbox_2d\": [173, 595, 259, 837], \"score\": 0.9}, {\"bbox_2d\": [265, 581, 315, 806], \"score\": 0.9}, {\"bbox_2d\": [310, 571, 351, 764], \"score\": 0.9}, {\"bbox_2d\": [338, 581, 387, 764], \"score\": 0.9}, {\"bbox_2d\": [449, 575, 492, 756], \"score\": 0.9}, {\"bbox_2d\": [475, 604, 542, 806], \"score\": 0.9}, {\"bbox_2d\": [668, 575, 743, 818], \"score\": 0.9}, {\"bbox_2d\": [740, 571, 783, 775], \"score\": 0.9}, {\"bbox_2d\": [771, 571, 814, 787], \"score\": 0.9}, {\"bbox_2d\": [822, 587, 851, 764], \"score\": 0.9}, {\"bbox_2d\": [847, 591, 921, 903], \"score\": 0.9}, {\"bbox_2d\": [902, 604, 959, 795], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 121, + 581, + 170, + 764 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 173, + 595, + 259, + 837 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 265, + 581, + 315, + 806 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 310, + 571, + 351, + 764 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 338, + 581, + 387, + 764 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 449, + 575, + 492, + 756 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 475, + 604, + 542, + 806 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 668, + 575, + 743, + 818 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 740, + 571, + 783, + 775 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 771, + 571, + 814, + 787 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 822, + 587, + 851, + 764 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 847, + 591, + 921, + 903 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 902, + 604, + 959, + 795 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 447, + "prompt_tokens": 600 + } + ] + }, + { + "sample_ids": [ + "testB-65935-181776" + ], + "batch_size": 1, + "generation_latency_seconds": 4.57236669503618, + "batch_wall_latency_seconds": 4.583186280098744, + "amortized_generation_ms_per_sample": 4572.36669503618, + "amortized_batch_wall_ms_per_sample": 4583.186280098744, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[347,333,951,761],\"score\":0.95},{\"bbox_2d\":[246,421,777,987],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 347, + 333, + 951, + 761 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 246, + 421, + 777, + 987 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 61, + "prompt_tokens": 564 + } + ] + }, + { + "sample_ids": [ + "testA-81622-224664" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6317770309979096, + "batch_wall_latency_seconds": 0.6445289449766278, + "amortized_generation_ms_per_sample": 631.7770309979096, + "amortized_batch_wall_ms_per_sample": 644.5289449766278, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 564 + } + ] + }, + { + "sample_ids": [ + "testA-10078-27739" + ], + "batch_size": 1, + "generation_latency_seconds": 2.393106485949829, + "batch_wall_latency_seconds": 2.4061070699244738, + "amortized_generation_ms_per_sample": 2393.106485949829, + "amortized_batch_wall_ms_per_sample": 2406.1070699244738, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[283,97,775,810],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 283, + 97, + 775, + 810 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 32, + "prompt_tokens": 564 + } + ] + }, + { + "sample_ids": [ + "testA-15582-42797" + ], + "batch_size": 1, + "generation_latency_seconds": 7.63340066303499, + "batch_wall_latency_seconds": 7.651124170981348, + "amortized_generation_ms_per_sample": 7633.40066303499, + "amortized_batch_wall_ms_per_sample": 7651.124170981348, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [85, 297, 446, 999], \"score\": 0.9}, {\"bbox_2d\": [315, 413, 542, 999], \"score\": 0.9}, {\"bbox_2d\": [542, 476, 642, 876], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 85, + 297, + 446, + 999 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 315, + 413, + 542, + 999 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 542, + 476, + 642, + 876 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 106, + "prompt_tokens": 619 + } + ] + }, + { + "sample_ids": [ + "testA-5038-13943" + ], + "batch_size": 1, + "generation_latency_seconds": 3.06427818199154, + "batch_wall_latency_seconds": 3.0877813909901306, + "amortized_generation_ms_per_sample": 3064.27818199154, + "amortized_batch_wall_ms_per_sample": 3087.7813909901306, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [521, 246, 793, 999], \"score\": 0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 521, + 246, + 793, + 999 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 40, + "prompt_tokens": 604 + } + ] + }, + { + "sample_ids": [ + "testA-30844-84861" + ], + "batch_size": 1, + "generation_latency_seconds": 2.516077253036201, + "batch_wall_latency_seconds": 2.5232016970403492, + "amortized_generation_ms_per_sample": 2516.077253036201, + "amortized_batch_wall_ms_per_sample": 2523.2016970403492, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[508,536,807,722],\"score\":0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 508, + 536, + 807, + 722 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 32, + "prompt_tokens": 623 + } + ] + }, + { + "sample_ids": [ + "testB-15419-42356" + ], + "batch_size": 1, + "generation_latency_seconds": 2.673266503959894, + "batch_wall_latency_seconds": 2.6889539459953085, + "amortized_generation_ms_per_sample": 2673.266503959894, + "amortized_batch_wall_ms_per_sample": 2688.9539459953085, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[107,722,587,999],\"score\":0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 107, + 722, + 587, + 999 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 32, + "prompt_tokens": 566 + } + ] + }, + { + "sample_ids": [ + "testB-27486-75580" + ], + "batch_size": 1, + "generation_latency_seconds": 4.617897133924998, + "batch_wall_latency_seconds": 4.632912510889582, + "amortized_generation_ms_per_sample": 4617.897133924998, + "amortized_batch_wall_ms_per_sample": 4632.912510889582, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[542,393,999,999],\"score\":0.95},{\"bbox_2d\":[646,271,911,471],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 542, + 393, + 999, + 999 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 646, + 271, + 911, + 471 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 61, + "prompt_tokens": 670 + } + ] + }, + { + "sample_ids": [ + "testB-945-2632" + ], + "batch_size": 1, + "generation_latency_seconds": 12.483138478011824, + "batch_wall_latency_seconds": 12.503725816030055, + "amortized_generation_ms_per_sample": 12483.138478011824, + "amortized_batch_wall_ms_per_sample": 12503.725816030055, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [213, 692, 283, 983], \"score\": 0.9}, {\"bbox_2d\": [321, 661, 415, 973], \"score\": 0.9}, {\"bbox_2d\": [473, 661, 555, 966], \"score\": 0.9}, {\"bbox_2d\": [638, 626, 713, 973], \"score\": 0.9}, {\"bbox_2d\": [775, 638, 871, 988], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 213, + 692, + 283, + 983 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 321, + 661, + 415, + 973 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 473, + 661, + 555, + 966 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 638, + 626, + 713, + 973 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 775, + 638, + 871, + 988 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 175, + "prompt_tokens": 670 + } + ] + }, + { + "sample_ids": [ + "testA-24438-67190" + ], + "batch_size": 1, + "generation_latency_seconds": 4.798777529038489, + "batch_wall_latency_seconds": 4.815012337057851, + "amortized_generation_ms_per_sample": 4798.777529038489, + "amortized_batch_wall_ms_per_sample": 4815.012337057851, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[731,375,999,999],\"score\":0.95},{\"bbox_2d\":[305,247,585,928],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 731, + 375, + 999, + 999 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 305, + 247, + 585, + 928 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 61, + "prompt_tokens": 605 + } + ] + }, + { + "sample_ids": [ + "testB-66161-182405" + ], + "batch_size": 1, + "generation_latency_seconds": 2.5812068269588053, + "batch_wall_latency_seconds": 2.593027200899087, + "amortized_generation_ms_per_sample": 2581.2068269588053, + "amortized_batch_wall_ms_per_sample": 2593.027200899087, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[103,193,755,713],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 103, + 193, + 755, + 713 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 33, + "prompt_tokens": 563 + } + ] + }, + { + "sample_ids": [ + "testB-42726-117659" + ], + "batch_size": 1, + "generation_latency_seconds": 3.0745598620269448, + "batch_wall_latency_seconds": 3.0872938209213316, + "amortized_generation_ms_per_sample": 3074.5598620269448, + "amortized_batch_wall_ms_per_sample": 3087.2938209213316, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [743, 458, 999, 878], \"score\": 0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 743, + 458, + 999, + 878 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 40, + "prompt_tokens": 604 + } + ] + }, + { + "sample_ids": [ + "testB-69439-191450" + ], + "batch_size": 1, + "generation_latency_seconds": 0.5888717719353735, + "batch_wall_latency_seconds": 0.5951321049360558, + "amortized_generation_ms_per_sample": 588.8717719353735, + "amortized_batch_wall_ms_per_sample": 595.1321049360558, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 492 + } + ] + }, + { + "sample_ids": [ + "testA-54307-149426" + ], + "batch_size": 1, + "generation_latency_seconds": 4.398384497035295, + "batch_wall_latency_seconds": 4.4204836380667984, + "amortized_generation_ms_per_sample": 4398.384497035295, + "amortized_batch_wall_ms_per_sample": 4420.483638066798, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[542,48,838,923],\"score\":0.95},{\"bbox_2d\":[177,0,567,477],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 542, + 48, + 838, + 923 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 177, + 0, + 567, + 477 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 58, + "prompt_tokens": 564 + } + ] + }, + { + "sample_ids": [ + "testB-37669-103831" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6747266949387267, + "batch_wall_latency_seconds": 0.699499664013274, + "amortized_generation_ms_per_sample": 674.7266949387267, + "amortized_batch_wall_ms_per_sample": 699.499664013274, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 648 + } + ] + }, + { + "sample_ids": [ + "testB-50844-140105" + ], + "batch_size": 1, + "generation_latency_seconds": 0.652067915070802, + "batch_wall_latency_seconds": 0.6656363360816613, + "amortized_generation_ms_per_sample": 652.067915070802, + "amortized_batch_wall_ms_per_sample": 665.6363360816613, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 561 + } + ] + }, + { + "sample_ids": [ + "testA-79438-218690" + ], + "batch_size": 1, + "generation_latency_seconds": 3.1018499380443245, + "batch_wall_latency_seconds": 3.1217426829971373, + "amortized_generation_ms_per_sample": 3101.8499380443245, + "amortized_batch_wall_ms_per_sample": 3121.7426829971373, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [255, 173, 536, 923], \"score\": 0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 255, + 173, + 536, + 923 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 40, + "prompt_tokens": 562 + } + ] + }, + { + "sample_ids": [ + "testA-62940-173454" + ], + "batch_size": 1, + "generation_latency_seconds": 3.057670062989928, + "batch_wall_latency_seconds": 3.073035439942032, + "amortized_generation_ms_per_sample": 3057.670062989928, + "amortized_batch_wall_ms_per_sample": 3073.035439942032, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [626, 266, 751, 693], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 626, + 266, + 751, + 693 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 39, + "prompt_tokens": 605 + } + ] + }, + { + "sample_ids": [ + "testA-81245-223632" + ], + "batch_size": 1, + "generation_latency_seconds": 10.252644294989295, + "batch_wall_latency_seconds": 10.266198730911128, + "amortized_generation_ms_per_sample": 10252.644294989295, + "amortized_batch_wall_ms_per_sample": 10266.198730911128, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [584, 117, 783, 882], \"score\": 0.9}, {\"bbox_2d\": [707, 131, 792, 761], \"score\": 0.9}, {\"bbox_2d\": [798, 105, 915, 655], \"score\": 0.9}, {\"bbox_2d\": [533, 178, 655, 705], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 584, + 117, + 783, + 882 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 707, + 131, + 792, + 761 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 798, + 105, + 915, + 655 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 533, + 178, + 655, + 705 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 141, + "prompt_tokens": 570 + } + ] + }, + { + "sample_ids": [ + "testA-47070-129599" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6474383240565658, + "batch_wall_latency_seconds": 0.6609540630597621, + "amortized_generation_ms_per_sample": 647.4383240565658, + "amortized_batch_wall_ms_per_sample": 660.9540630597621, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 560 + } + ] + }, + { + "sample_ids": [ + "testB-12381-34098" + ], + "batch_size": 1, + "generation_latency_seconds": 6.267876331927255, + "batch_wall_latency_seconds": 6.293664561933838, + "amortized_generation_ms_per_sample": 6267.876331927255, + "amortized_batch_wall_ms_per_sample": 6293.664561933838, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[458,242,771,473],\"score\":0.9},{\"bbox_2d\":[298,343,826,817],\"score\":0.9},{\"bbox_2d\":[305,466,793,840],\"score\":0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 458, + 242, + 771, + 473 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 298, + 343, + 826, + 817 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 305, + 466, + 793, + 840 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 86, + "prompt_tokens": 574 + } + ] + }, + { + "sample_ids": [ + "testA-44142-121498" + ], + "batch_size": 1, + "generation_latency_seconds": 5.1425224100239575, + "batch_wall_latency_seconds": 5.153767512994818, + "amortized_generation_ms_per_sample": 5142.5224100239575, + "amortized_batch_wall_ms_per_sample": 5153.767512994818, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [0, 95, 585, 987], \"score\": 0.9}, {\"bbox_2d\": [567, 126, 999, 987], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 95, + 585, + 987 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 567, + 126, + 999, + 987 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 70, + "prompt_tokens": 563 + } + ] + }, + { + "sample_ids": [ + "testA-73258-201893" + ], + "batch_size": 1, + "generation_latency_seconds": 2.4450040519004688, + "batch_wall_latency_seconds": 2.4576197939459234, + "amortized_generation_ms_per_sample": 2445.0040519004688, + "amortized_batch_wall_ms_per_sample": 2457.6197939459234, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[401,274,999,987],\"score\":0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 401, + 274, + 999, + 987 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 32, + "prompt_tokens": 606 + } + ] + }, + { + "sample_ids": [ + "testB-46644-128436" + ], + "batch_size": 1, + "generation_latency_seconds": 4.464105351013131, + "batch_wall_latency_seconds": 4.473494493984617, + "amortized_generation_ms_per_sample": 4464.105351013131, + "amortized_batch_wall_ms_per_sample": 4473.494493984617, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[10,183,537,826],\"score\":0.95},{\"bbox_2d\":[483,0,999,743],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 10, + 183, + 537, + 826 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 483, + 0, + 999, + 743 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 58, + "prompt_tokens": 497 + } + ] + }, + { + "sample_ids": [ + "testA-43889-120792" + ], + "batch_size": 1, + "generation_latency_seconds": 3.0225026920670643, + "batch_wall_latency_seconds": 3.039261986967176, + "amortized_generation_ms_per_sample": 3022.5026920670643, + "amortized_batch_wall_ms_per_sample": 3039.261986967176, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [305, 303, 475, 882], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 305, + 303, + 475, + 882 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 39, + "prompt_tokens": 571 + } + ] + }, + { + "sample_ids": [ + "testB-36082-99363" + ], + "batch_size": 1, + "generation_latency_seconds": 4.3171981680206954, + "batch_wall_latency_seconds": 4.324737289105542, + "amortized_generation_ms_per_sample": 4317.198168020695, + "amortized_batch_wall_ms_per_sample": 4324.737289105542, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0,107,417,876],\"score\":0.95},{\"bbox_2d\":[442,0,999,986],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 107, + 417, + 876 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 442, + 0, + 999, + 986 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 57, + "prompt_tokens": 412 + } + ] + }, + { + "sample_ids": [ + "testB-33108-91152" + ], + "batch_size": 1, + "generation_latency_seconds": 2.9848218420520425, + "batch_wall_latency_seconds": 2.9953485280275345, + "amortized_generation_ms_per_sample": 2984.8218420520425, + "amortized_batch_wall_ms_per_sample": 2995.3485280275345, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [58, 201, 385, 458], \"score\": 0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 58, + 201, + 385, + 458 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 39, + "prompt_tokens": 602 + } + ] + }, + { + "sample_ids": [ + "testA-26856-73830" + ], + "batch_size": 1, + "generation_latency_seconds": 3.0266257260227576, + "batch_wall_latency_seconds": 3.0415181359276175, + "amortized_generation_ms_per_sample": 3026.6257260227576, + "amortized_batch_wall_ms_per_sample": 3041.5181359276175, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [576, 343, 783, 713], \"score\": 0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 576, + 343, + 783, + 713 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 40, + "prompt_tokens": 562 + } + ] + }, + { + "sample_ids": [ + "testA-81271-223709" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6367205450078472, + "batch_wall_latency_seconds": 0.6504157679155469, + "amortized_generation_ms_per_sample": 636.7205450078472, + "amortized_batch_wall_ms_per_sample": 650.4157679155469, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 560 + } + ] + }, + { + "sample_ids": [ + "testA-54554-150141" + ], + "batch_size": 1, + "generation_latency_seconds": 11.380541390972212, + "batch_wall_latency_seconds": 11.400562496040948, + "amortized_generation_ms_per_sample": 11380.541390972212, + "amortized_batch_wall_ms_per_sample": 11400.562496040948, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[27,0,121,243],\"score\":0.9},{\"bbox_2d\":[62,0,185,258],\"score\":0.9},{\"bbox_2d\":[142,0,222,255],\"score\":0.9},{\"bbox_2d\":[185,0,283,261],\"score\":0.9},{\"bbox_2d\":[277,312,597,826],\"score\":0.9},{\"bbox_2d\":[655,500,951,871],\"score\":0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 27, + 0, + 121, + 243 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 62, + 0, + 185, + 258 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 142, + 0, + 222, + 255 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 185, + 0, + 283, + 261 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 277, + 312, + 597, + 826 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 655, + 500, + 951, + 871 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 157, + "prompt_tokens": 610 + } + ] + }, + { + "sample_ids": [ + "testB-38636-106481" + ], + "batch_size": 1, + "generation_latency_seconds": 4.233924768050201, + "batch_wall_latency_seconds": 4.251373155973852, + "amortized_generation_ms_per_sample": 4233.924768050201, + "amortized_batch_wall_ms_per_sample": 4251.373155973852, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[494,177,881,985],\"score\":0.95},{\"bbox_2d\":[0,42,307,722],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 494, + 177, + 881, + 985 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 0, + 42, + 307, + 722 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 58, + "prompt_tokens": 469 + } + ] + }, + { + "sample_ids": [ + "testA-5734-15837" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6504808579338714, + "batch_wall_latency_seconds": 0.6668235529214144, + "amortized_generation_ms_per_sample": 650.4808579338714, + "amortized_batch_wall_ms_per_sample": 666.8235529214144, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 565 + } + ] + }, + { + "sample_ids": [ + "testB-67558-186325" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6280788460280746, + "batch_wall_latency_seconds": 0.6436845100251958, + "amortized_generation_ms_per_sample": 628.0788460280746, + "amortized_batch_wall_ms_per_sample": 643.6845100251958, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 600 + } + ] + }, + { + "sample_ids": [ + "testB-47053-129554" + ], + "batch_size": 1, + "generation_latency_seconds": 4.556327463011257, + "batch_wall_latency_seconds": 4.573514219024219, + "amortized_generation_ms_per_sample": 4556.327463011257, + "amortized_batch_wall_ms_per_sample": 4573.514219024219, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[438,321,871,542],\"score\":0.95},{\"bbox_2d\":[438,452,851,863],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 438, + 321, + 871, + 542 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 438, + 452, + 851, + 863 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 61, + "prompt_tokens": 604 + } + ] + }, + { + "sample_ids": [ + "testA-93064-256422" + ], + "batch_size": 1, + "generation_latency_seconds": 2.830894450074993, + "batch_wall_latency_seconds": 2.8475917810574174, + "amortized_generation_ms_per_sample": 2830.894450074993, + "amortized_batch_wall_ms_per_sample": 2847.5917810574174, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [0, 226, 173, 755], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 226, + 173, + 755 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 37, + "prompt_tokens": 549 + } + ] + }, + { + "sample_ids": [ + "testB-60557-166821" + ], + "batch_size": 1, + "generation_latency_seconds": 5.58066594589036, + "batch_wall_latency_seconds": 5.594979963963851, + "amortized_generation_ms_per_sample": 5580.66594589036, + "amortized_batch_wall_ms_per_sample": 5594.979963963851, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [857, 442, 875, 477], \"score\": 0.9}, {\"bbox_2d\": [347, 442, 365, 477], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 857, + 442, + 875, + 477 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 347, + 442, + 365, + 477 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 73, + "prompt_tokens": 500 + } + ] + }, + { + "sample_ids": [ + "testA-55116-151688" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6620599570451304, + "batch_wall_latency_seconds": 0.6773106920300052, + "amortized_generation_ms_per_sample": 662.0599570451304, + "amortized_batch_wall_ms_per_sample": 677.3106920300052, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 600 + } + ] + }, + { + "sample_ids": [ + "testB-65796-181392" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6342472250107676, + "batch_wall_latency_seconds": 0.6483092680573463, + "amortized_generation_ms_per_sample": 634.2472250107676, + "amortized_batch_wall_ms_per_sample": 648.3092680573463, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 581 + } + ] + }, + { + "sample_ids": [ + "testB-23014-63278" + ], + "batch_size": 1, + "generation_latency_seconds": 2.7931598050054163, + "batch_wall_latency_seconds": 2.8087078949902207, + "amortized_generation_ms_per_sample": 2793.1598050054163, + "amortized_batch_wall_ms_per_sample": 2808.7078949902207, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[33, 303, 466, 761],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 33, + 303, + 466, + 761 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 35, + "prompt_tokens": 563 + } + ] + }, + { + "sample_ids": [ + "testA-24562-67515" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6919117919169366, + "batch_wall_latency_seconds": 0.7147108289645985, + "amortized_generation_ms_per_sample": 691.9117919169366, + "amortized_batch_wall_ms_per_sample": 714.7108289645985, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 606 + } + ] + }, + { + "sample_ids": [ + "testA-17736-48684" + ], + "batch_size": 1, + "generation_latency_seconds": 3.0041811569826677, + "batch_wall_latency_seconds": 3.0170002209488302, + "amortized_generation_ms_per_sample": 3004.1811569826677, + "amortized_batch_wall_ms_per_sample": 3017.0002209488302, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [508, 426, 765, 983], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 508, + 426, + 765, + 983 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 39, + "prompt_tokens": 605 + } + ] + }, + { + "sample_ids": [ + "testA-58475-160949" + ], + "batch_size": 1, + "generation_latency_seconds": 2.428418811992742, + "batch_wall_latency_seconds": 2.442452701041475, + "amortized_generation_ms_per_sample": 2428.418811992742, + "amortized_batch_wall_ms_per_sample": 2442.452701041475, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[46,36,576,999],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 46, + 36, + 576, + 999 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 31, + "prompt_tokens": 564 + } + ] + }, + { + "sample_ids": [ + "testA-80455-221493" + ], + "batch_size": 1, + "generation_latency_seconds": 2.4254801200004295, + "batch_wall_latency_seconds": 2.4414882709970698, + "amortized_generation_ms_per_sample": 2425.4801200004295, + "amortized_batch_wall_ms_per_sample": 2441.48827099707, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0,483,581,938],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 483, + 581, + 938 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 31, + "prompt_tokens": 541 + } + ] + }, + { + "sample_ids": [ + "testA-31763-87405" + ], + "batch_size": 1, + "generation_latency_seconds": 7.767380379955284, + "batch_wall_latency_seconds": 7.780358829069883, + "amortized_generation_ms_per_sample": 7767.380379955284, + "amortized_batch_wall_ms_per_sample": 7780.358829069883, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [626, 737, 652, 821], \"score\": 0.9}, {\"bbox_2d\": [426, 913, 452, 972], \"score\": 0.9}, {\"bbox_2d\": [356, 852, 378, 936], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 626, + 737, + 652, + 821 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 426, + 913, + 452, + 972 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 356, + 852, + 378, + 936 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 107, + "prompt_tokens": 445 + } + ] + }, + { + "sample_ids": [ + "testB-23920-65742" + ], + "batch_size": 1, + "generation_latency_seconds": 2.9294577399268746, + "batch_wall_latency_seconds": 2.9447807769756764, + "amortized_generation_ms_per_sample": 2929.4577399268746, + "amortized_batch_wall_ms_per_sample": 2944.7807769756764, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [537, 417, 999, 993], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 537, + 417, + 999, + 993 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 39, + "prompt_tokens": 568 + } + ] + }, + { + "sample_ids": [ + "testA-469-1309" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6524032501038164, + "batch_wall_latency_seconds": 0.6627495769644156, + "amortized_generation_ms_per_sample": 652.4032501038164, + "amortized_batch_wall_ms_per_sample": 662.7495769644156, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 621 + } + ] + }, + { + "sample_ids": [ + "testB-56691-155998" + ], + "batch_size": 1, + "generation_latency_seconds": 0.655415665009059, + "batch_wall_latency_seconds": 0.67216857499443, + "amortized_generation_ms_per_sample": 655.415665009059, + "amortized_batch_wall_ms_per_sample": 672.16857499443, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 563 + } + ] + }, + { + "sample_ids": [ + "testB-31493-86668" + ], + "batch_size": 1, + "generation_latency_seconds": 4.518256991053931, + "batch_wall_latency_seconds": 4.5385722850915045, + "amortized_generation_ms_per_sample": 4518.256991053931, + "amortized_batch_wall_ms_per_sample": 4538.572285091504, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[537,426,747,765],\"score\":0.95},{\"bbox_2d\":[152,426,466,462],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 537, + 426, + 747, + 765 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 152, + 426, + 466, + 462 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 61, + "prompt_tokens": 614 + } + ] + }, + { + "sample_ids": [ + "testA-59284-163254" + ], + "batch_size": 1, + "generation_latency_seconds": 5.580998525954783, + "batch_wall_latency_seconds": 5.5959404041059315, + "amortized_generation_ms_per_sample": 5580.998525954783, + "amortized_batch_wall_ms_per_sample": 5595.9404041059315, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [501, 381, 696, 999], \"score\": 0.9}, {\"bbox_2d\": [561, 300, 814, 426], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 501, + 381, + 696, + 999 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 561, + 300, + 814, + 426 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 73, + "prompt_tokens": 623 + } + ] + }, + { + "sample_ids": [ + "testA-57397-158015" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6980972309829667, + "batch_wall_latency_seconds": 0.709314426057972, + "amortized_generation_ms_per_sample": 698.0972309829667, + "amortized_batch_wall_ms_per_sample": 709.314426057972, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 602 + } + ] + }, + { + "sample_ids": [ + "testB-2935-8111" + ], + "batch_size": 1, + "generation_latency_seconds": 4.602234860998578, + "batch_wall_latency_seconds": 4.6172516259830445, + "amortized_generation_ms_per_sample": 4602.234860998578, + "amortized_batch_wall_ms_per_sample": 4617.2516259830445, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[371,91,707,431],\"score\":0.95},{\"bbox_2d\":[585,181,999,705],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 371, + 91, + 707, + 431 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 585, + 181, + 999, + 705 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 60, + "prompt_tokens": 570 + } + ] + }, + { + "sample_ids": [ + "testB-22290-61226" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6584999479819089, + "batch_wall_latency_seconds": 0.6694054200779647, + "amortized_generation_ms_per_sample": 658.4999479819089, + "amortized_batch_wall_ms_per_sample": 669.4054200779647, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 586 + } + ] + }, + { + "sample_ids": [ + "testB-8416-23140" + ], + "batch_size": 1, + "generation_latency_seconds": 6.426246937015094, + "batch_wall_latency_seconds": 6.434293171972968, + "amortized_generation_ms_per_sample": 6426.246937015094, + "amortized_batch_wall_ms_per_sample": 6434.293171972968, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0,292,295,642],\"score\":0.95},{\"bbox_2d\":[234,252,581,727],\"score\":0.95},{\"bbox_2d\":[722,183,959,808],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 292, + 295, + 642 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 234, + 252, + 581, + 727 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 722, + 183, + 959, + 808 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 87, + "prompt_tokens": 563 + } + ] + }, + { + "sample_ids": [ + "testA-60731-167320" + ], + "batch_size": 1, + "generation_latency_seconds": 5.329908382031135, + "batch_wall_latency_seconds": 5.342693028971553, + "amortized_generation_ms_per_sample": 5329.908382031135, + "amortized_batch_wall_ms_per_sample": 5342.693028971553, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [0, 81, 446, 987], \"score\": 0.9}, {\"bbox_2d\": [483, 0, 792, 201], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 81, + 446, + 987 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 483, + 0, + 792, + 201 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 68, + "prompt_tokens": 606 + } + ] + }, + { + "sample_ids": [ + "testA-49055-135086" + ], + "batch_size": 1, + "generation_latency_seconds": 4.7276995739666745, + "batch_wall_latency_seconds": 4.746967880986631, + "amortized_generation_ms_per_sample": 4727.6995739666745, + "amortized_batch_wall_ms_per_sample": 4746.967880986631, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[615,242,851,999],\"score\":0.95},{\"bbox_2d\":[137,142,326,999],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 615, + 242, + 851, + 999 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 137, + 142, + 326, + 999 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 61, + "prompt_tokens": 566 + } + ] + }, + { + "sample_ids": [ + "testA-7857-21644" + ], + "batch_size": 1, + "generation_latency_seconds": 2.446160126943141, + "batch_wall_latency_seconds": 2.4602624271064997, + "amortized_generation_ms_per_sample": 2446.160126943141, + "amortized_batch_wall_ms_per_sample": 2460.2624271064997, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[668,36,999,999],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 668, + 36, + 999, + 999 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 32, + "prompt_tokens": 607 + } + ] + }, + { + "sample_ids": [ + "testB-79645-219273" + ], + "batch_size": 1, + "generation_latency_seconds": 15.058248669025488, + "batch_wall_latency_seconds": 15.065027808072045, + "amortized_generation_ms_per_sample": 15058.248669025488, + "amortized_batch_wall_ms_per_sample": 15065.027808072045, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [542, 417, 798, 722], \"score\": 0.9}, {\"bbox_2d\": [295, 421, 552, 715], \"score\": 0.9}, {\"bbox_2d\": [147, 452, 310, 665], \"score\": 0.9}, {\"bbox_2d\": [55, 476, 115, 645], \"score\": 0.9}, {\"bbox_2d\": [0, 448, 62, 677], \"score\": 0.9}, {\"bbox_2d\": [113, 478, 162, 595], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 542, + 417, + 798, + 722 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 295, + 421, + 552, + 715 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 147, + 452, + 310, + 665 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 55, + 476, + 115, + 645 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 0, + 448, + 62, + 677 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 113, + 478, + 162, + 595 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 205, + "prompt_tokens": 601 + } + ] + }, + { + "sample_ids": [ + "testB-52921-145707" + ], + "batch_size": 1, + "generation_latency_seconds": 0.65654409409035, + "batch_wall_latency_seconds": 0.6844046280020848, + "amortized_generation_ms_per_sample": 656.54409409035, + "amortized_batch_wall_ms_per_sample": 684.4046280020848, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 571 + } + ] + }, + { + "sample_ids": [ + "testB-69479-191562" + ], + "batch_size": 1, + "generation_latency_seconds": 2.294087173999287, + "batch_wall_latency_seconds": 2.3162708869203925, + "amortized_generation_ms_per_sample": 2294.087173999287, + "amortized_batch_wall_ms_per_sample": 2316.2708869203925, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0,31,972,999],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 31, + 972, + 999 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 30, + "prompt_tokens": 561 + } + ] + }, + { + "sample_ids": [ + "testA-69466-191524" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6702312079723924, + "batch_wall_latency_seconds": 0.683375352062285, + "amortized_generation_ms_per_sample": 670.2312079723924, + "amortized_batch_wall_ms_per_sample": 683.375352062285, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 602 + } + ] + } + ], + "mean_generation_ms_per_sample": 4056.054782922729, + "median_generation_ms_per_sample": 3024.564209044911, + "mean_batch_wall_ms_per_sample": 4071.4221567668574, + "median_batch_wall_ms_per_sample": 3040.390061447397, + "mean_prompt_tokens": 578.6875, + "mean_completion_tokens": 53.203125, + "latency_scope": "A800 vLLM-0.18.0 native generation latency with the frozen task prompt and structured-output schema; model loading and warmup excluded. Batch wall time includes the real request/image preparation path.", + "flops_scope": "This file measures latency only. Counted FLOPs are measured separately by the HF reference probe and are not claimed to be complete vLLM kernel FLOPs." +} \ No newline at end of file diff --git a/paper_results/data/main_table_cost_qwen_20260912/components/qwen_latency.json b/paper_results/data/main_table_cost_qwen_20260912/components/qwen_latency.json new file mode 100644 index 0000000000000000000000000000000000000000..d4d3a8551f8fc5c964291e5c4768033d935c2b3e --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/components/qwen_latency.json @@ -0,0 +1,3066 @@ +{ + "status": "complete", + "model_kind": "Qwen3.8", + "task": "construction", + "expert": "yolo26x", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/Qwen3.8-27B-FP8", + "device": "cuda:0", + "gpu_memory_utilization": 0.85, + "sample_count_requested": 66, + "warmup_count": 2, + "measured_count": 64, + "batch_size": 1, + "sample_ids": [ + "0000894", + "0003510", + "0002450", + "0003360", + "0001470", + "0004484", + "0004166", + "0002167", + "0000380", + "0000516", + "0000176", + "0000943", + "0002853", + "0003176", + "0002495", + "0000756", + "0004213", + "0003472", + "0004290", + "0001443", + "0003365", + "0003782", + "0002089", + "0001710", + "0002148", + "0001289", + "0002751", + "0001252", + "0001922", + "0003581", + "0001233", + "0003383", + "0003266", + "0000824", + "0002144", + "0001450", + "0003429", + "0000205", + "0004209", + "0003623", + "0002496", + "0004049", + "0001483", + "0004197", + "0003082", + "0000758", + "0000564", + "0001580", + "0002136", + "0000949", + "0003045", + "0000013", + "0003889", + "0003261", + "0001629", + "0001569", + "0002536", + "0003053", + "0002661", + "0001664", + "0001351", + "0000304", + "0004537", + "0003801", + "0004321", + "0001878" + ], + "batches": [ + { + "sample_ids": [ + "0002450" + ], + "batch_size": 1, + "generation_latency_seconds": 10.705359333078377, + "batch_wall_latency_seconds": 10.720781230949797, + "amortized_generation_ms_per_sample": 10705.359333078377, + "amortized_batch_wall_ms_per_sample": 10720.781230949797, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker standing in the middle of the site is clearly bareheaded, missing a hard hat.\", \"boxes\": [[518, 668, 532, 745]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker standing in the middle of the site is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 518, + 668, + 532, + 745 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 147, + "prompt_tokens": 1473 + } + ] + }, + { + "sample_ids": [ + "0003360" + ], + "batch_size": 1, + "generation_latency_seconds": 11.7157509700628, + "batch_wall_latency_seconds": 11.733416861039586, + "amortized_generation_ms_per_sample": 11715.7509700628, + "amortized_batch_wall_ms_per_sample": 11733.416861039586, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The edge of the excavation is visible on the right side of the image, bordering a deep drop with no guardrails or fencing.\", \"boxes\": [[590, 315, 1000, 495]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The edge of the excavation is visible on the right side of the image, bordering a deep drop with no guardrails or fencing.", + "boxes": [ + [ + 590, + 315, + 1000, + 495 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 158, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001470" + ], + "batch_size": 1, + "generation_latency_seconds": 11.342743383022025, + "batch_wall_latency_seconds": 11.35942490492016, + "amortized_generation_ms_per_sample": 11342.743383022025, + "amortized_batch_wall_ms_per_sample": 11359.42490492016, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker on the ground is clearly visible without a hard hat.\", \"boxes\": [[560, 849, 582, 905]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker on the ground is clearly visible without a hard hat.", + "boxes": [ + [ + 560, + 849, + 582, + 905 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 141, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004484" + ], + "batch_size": 1, + "generation_latency_seconds": 13.103367771022022, + "batch_wall_latency_seconds": 13.121180747984909, + "amortized_generation_ms_per_sample": 13103.367771022022, + "amortized_batch_wall_ms_per_sample": 13121.180747984909, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": true, \"reason\": \"Workers are standing directly beside the tracks and rear of the yellow excavator, within its normal swing and travel zone.\", \"boxes\": [[608, 505, 640, 655], [658, 502, 690, 646]]}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": true, + "reason": "Workers are standing directly beside the tracks and rear of the yellow excavator, within its normal swing and travel zone.", + "boxes": [ + [ + 608, + 505, + 640, + 655 + ], + [ + 658, + 502, + 690, + 646 + ] + ] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 171, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004166" + ], + "batch_size": 1, + "generation_latency_seconds": 11.62961833900772, + "batch_wall_latency_seconds": 11.636370092979632, + "amortized_generation_ms_per_sample": 11629.61833900772, + "amortized_batch_wall_ms_per_sample": 11636.370092979632, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The edge of the concrete structure where the workers are standing lacks guardrails or fencing, bordering a deep drop/excavation area.\", \"boxes\": [[0, 620, 510, 835]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The edge of the concrete structure where the workers are standing lacks guardrails or fencing, bordering a deep drop/excavation area.", + "boxes": [ + [ + 0, + 620, + 510, + 835 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 154, + "prompt_tokens": 1129 + } + ] + }, + { + "sample_ids": [ + "0002167" + ], + "batch_size": 1, + "generation_latency_seconds": 11.212190569960512, + "batch_wall_latency_seconds": 11.235628957045265, + "amortized_generation_ms_per_sample": 11212.190569960512, + "amortized_batch_wall_ms_per_sample": 11235.628957045265, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The edge of the concrete platform borders a deep excavation area without guardrails or fencing, creating a significant fall hazard.\", \"boxes\": [[0, 315, 515, 620]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The edge of the concrete platform borders a deep excavation area without guardrails or fencing, creating a significant fall hazard.", + "boxes": [ + [ + 0, + 315, + 515, + 620 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 151, + "prompt_tokens": 1405 + } + ] + }, + { + "sample_ids": [ + "0000380" + ], + "batch_size": 1, + "generation_latency_seconds": 8.91931619006209, + "batch_wall_latency_seconds": 8.942373727913946, + "amortized_generation_ms_per_sample": 8919.31619006209, + "amortized_batch_wall_ms_per_sample": 8942.373727913946, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1683 + } + ] + }, + { + "sample_ids": [ + "0000516" + ], + "batch_size": 1, + "generation_latency_seconds": 8.323322776006535, + "batch_wall_latency_seconds": 8.3395675040083, + "amortized_generation_ms_per_sample": 8323.322776006535, + "amortized_batch_wall_ms_per_sample": 8339.5675040083, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1405 + } + ] + }, + { + "sample_ids": [ + "0000176" + ], + "batch_size": 1, + "generation_latency_seconds": 8.365567820030265, + "batch_wall_latency_seconds": 8.383442353922874, + "amortized_generation_ms_per_sample": 8365.567820030265, + "amortized_batch_wall_ms_per_sample": 8383.442353922874, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000943" + ], + "batch_size": 1, + "generation_latency_seconds": 8.261911686044186, + "batch_wall_latency_seconds": 8.284466639976017, + "amortized_generation_ms_per_sample": 8261.911686044186, + "amortized_batch_wall_ms_per_sample": 8284.466639976017, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002853" + ], + "batch_size": 1, + "generation_latency_seconds": 10.991469252971001, + "batch_wall_latency_seconds": 11.013957520015538, + "amortized_generation_ms_per_sample": 10991.469252971001, + "amortized_batch_wall_ms_per_sample": 11013.957520015538, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The upper edge of the deep excavation in the foreground lacks guardrails or fencing, exposing a significant drop.\", \"boxes\": [[0, 580, 998, 998]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The upper edge of the deep excavation in the foreground lacks guardrails or fencing, exposing a significant drop.", + "boxes": [ + [ + 0, + 580, + 998, + 998 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 147, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003176" + ], + "batch_size": 1, + "generation_latency_seconds": 10.860358019010164, + "batch_wall_latency_seconds": 10.878267464926466, + "amortized_generation_ms_per_sample": 10860.358019010164, + "amortized_batch_wall_ms_per_sample": 10878.267464926466, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The top edge of the deep excavation on the left side lacks guardrails or fencing, exposing a significant drop.\", \"boxes\": [[0, 500, 640, 998]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The top edge of the deep excavation on the left side lacks guardrails or fencing, exposing a significant drop.", + "boxes": [ + [ + 0, + 500, + 640, + 998 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 148, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002495" + ], + "batch_size": 1, + "generation_latency_seconds": 10.56970894197002, + "batch_wall_latency_seconds": 10.593795233988203, + "amortized_generation_ms_per_sample": 10569.70894197002, + "amortized_batch_wall_ms_per_sample": 10593.795233988203, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker crouching on the ground is clearly bareheaded, missing a hard hat.\", \"boxes\": [[475, 525, 571, 688]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker crouching on the ground is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 475, + 525, + 571, + 688 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 146, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000756" + ], + "batch_size": 1, + "generation_latency_seconds": 8.038495411048643, + "batch_wall_latency_seconds": 8.052992713986896, + "amortized_generation_ms_per_sample": 8038.495411048643, + "amortized_batch_wall_ms_per_sample": 8052.992713986896, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004213" + ], + "batch_size": 1, + "generation_latency_seconds": 8.011779504013248, + "batch_wall_latency_seconds": 8.047042324091308, + "amortized_generation_ms_per_sample": 8011.779504013248, + "amortized_batch_wall_ms_per_sample": 8047.042324091308, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003472" + ], + "batch_size": 1, + "generation_latency_seconds": 8.286275054095313, + "batch_wall_latency_seconds": 8.304093755898066, + "amortized_generation_ms_per_sample": 8286.275054095313, + "amortized_batch_wall_ms_per_sample": 8304.093755898066, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004290" + ], + "batch_size": 1, + "generation_latency_seconds": 8.341199293034151, + "batch_wall_latency_seconds": 8.358467961079441, + "amortized_generation_ms_per_sample": 8341.199293034151, + "amortized_batch_wall_ms_per_sample": 8358.467961079441, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001443" + ], + "batch_size": 1, + "generation_latency_seconds": 15.180880905943923, + "batch_wall_latency_seconds": 15.206637792987749, + "amortized_generation_ms_per_sample": 15180.880905943923, + "amortized_batch_wall_ms_per_sample": 15206.637792987749, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker on the left side of the image, standing on the ground near the scaffolding, is clearly bareheaded and not wearing a hard hat.\", \"boxes\": [[238, 458, 278, 688]]}, \"rule_2_fall_protection_violation\": {\"present\": true, \"reason\": \"A worker is standing on a low concrete beam/ledge without a safety harness, exposed to a fall from a height of approximately 3 meters or more.\", \"boxes\": [[315, 325, 368, 505]]}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker on the left side of the image, standing on the ground near the scaffolding, is clearly bareheaded and not wearing a hard hat.", + "boxes": [ + [ + 238, + 458, + 278, + 688 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": true, + "reason": "A worker is standing on a low concrete beam/ledge without a safety harness, exposed to a fall from a height of approximately 3 meters or more.", + "boxes": [ + [ + 315, + 325, + 368, + 505 + ] + ] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 207, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003365" + ], + "batch_size": 1, + "generation_latency_seconds": 8.154706532950513, + "batch_wall_latency_seconds": 8.171265691053122, + "amortized_generation_ms_per_sample": 8154.706532950513, + "amortized_batch_wall_ms_per_sample": 8171.265691053122, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003782" + ], + "batch_size": 1, + "generation_latency_seconds": 8.263969205901958, + "batch_wall_latency_seconds": 8.278776310035028, + "amortized_generation_ms_per_sample": 8263.969205901958, + "amortized_batch_wall_ms_per_sample": 8278.776310035028, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002089" + ], + "batch_size": 1, + "generation_latency_seconds": 8.21782910998445, + "batch_wall_latency_seconds": 8.233122498961166, + "amortized_generation_ms_per_sample": 8217.82910998445, + "amortized_batch_wall_ms_per_sample": 8233.122498961166, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1531 + } + ] + }, + { + "sample_ids": [ + "0001710" + ], + "batch_size": 1, + "generation_latency_seconds": 8.018760634935461, + "batch_wall_latency_seconds": 8.032283170963638, + "amortized_generation_ms_per_sample": 8018.760634935461, + "amortized_batch_wall_ms_per_sample": 8032.2831709636375, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1405 + } + ] + }, + { + "sample_ids": [ + "0002148" + ], + "batch_size": 1, + "generation_latency_seconds": 10.378318366943859, + "batch_wall_latency_seconds": 10.404688362963498, + "amortized_generation_ms_per_sample": 10378.318366943859, + "amortized_batch_wall_ms_per_sample": 10404.688362963498, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker in the middle distance is clearly bareheaded, missing a hard hat.\", \"boxes\": [[561, 300, 582, 408]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker in the middle distance is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 561, + 300, + 582, + 408 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 144, + "prompt_tokens": 1531 + } + ] + }, + { + "sample_ids": [ + "0001289" + ], + "batch_size": 1, + "generation_latency_seconds": 11.48702211806085, + "batch_wall_latency_seconds": 11.514577882946469, + "amortized_generation_ms_per_sample": 11487.02211806085, + "amortized_batch_wall_ms_per_sample": 11514.577882946469, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The upper edge of the deep excavation on the right side lacks guardrails or fencing, bordering a steep drop of several meters.\", \"boxes\": [[430, 240, 998, 480]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The upper edge of the deep excavation on the right side lacks guardrails or fencing, bordering a steep drop of several meters.", + "boxes": [ + [ + 430, + 240, + 998, + 480 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 156, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002751" + ], + "batch_size": 1, + "generation_latency_seconds": 8.446506896987557, + "batch_wall_latency_seconds": 8.467677644919604, + "amortized_generation_ms_per_sample": 8446.506896987557, + "amortized_batch_wall_ms_per_sample": 8467.677644919604, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001252" + ], + "batch_size": 1, + "generation_latency_seconds": 8.315838309004903, + "batch_wall_latency_seconds": 8.333076176000759, + "amortized_generation_ms_per_sample": 8315.838309004903, + "amortized_batch_wall_ms_per_sample": 8333.076176000759, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001922" + ], + "batch_size": 1, + "generation_latency_seconds": 8.236222366103902, + "batch_wall_latency_seconds": 8.253340549999848, + "amortized_generation_ms_per_sample": 8236.222366103902, + "amortized_batch_wall_ms_per_sample": 8253.340549999848, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1107 + } + ] + }, + { + "sample_ids": [ + "0003581" + ], + "batch_size": 1, + "generation_latency_seconds": 8.227389317005873, + "batch_wall_latency_seconds": 8.240366365062073, + "amortized_generation_ms_per_sample": 8227.389317005873, + "amortized_batch_wall_ms_per_sample": 8240.366365062073, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1694 + } + ] + }, + { + "sample_ids": [ + "0001233" + ], + "batch_size": 1, + "generation_latency_seconds": 11.756323368987069, + "batch_wall_latency_seconds": 11.764942721929401, + "amortized_generation_ms_per_sample": 11756.323368987069, + "amortized_batch_wall_ms_per_sample": 11764.942721929401, + "outputs": [ + { + "raw_response": "{\n \"events\": {\n \"rule_1_ppe_violation\": {\n \"present\": false,\n \"reason\": null,\n \"boxes\": []\n },\n \"rule_2_fall_protection_violation\": {\n \"present\": false,\n \"reason\": null,\n \"boxes\": []\n },\n \"rule_3_unprotected_edge_violation\": {\n \"present\": false,\n \"reason\": null,\n \"boxes\": []\n },\n \"rule_4_excavator_proximity_violation\": {\n \"present\": false,\n \"reason\": null,\n \"boxes\": []\n }\n }\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 163, + "prompt_tokens": 1453 + } + ] + }, + { + "sample_ids": [ + "0003383" + ], + "batch_size": 1, + "generation_latency_seconds": 8.106548791052774, + "batch_wall_latency_seconds": 8.117786544025876, + "amortized_generation_ms_per_sample": 8106.548791052774, + "amortized_batch_wall_ms_per_sample": 8117.786544025876, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1573 + } + ] + }, + { + "sample_ids": [ + "0003266" + ], + "batch_size": 1, + "generation_latency_seconds": 8.274491983000189, + "batch_wall_latency_seconds": 8.288345888955519, + "amortized_generation_ms_per_sample": 8274.491983000189, + "amortized_batch_wall_ms_per_sample": 8288.345888955519, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000824" + ], + "batch_size": 1, + "generation_latency_seconds": 11.22929672000464, + "batch_wall_latency_seconds": 11.251729905954562, + "amortized_generation_ms_per_sample": 11229.29672000464, + "amortized_batch_wall_ms_per_sample": 11251.729905954562, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": true, \"reason\": \"A worker is standing on the ground within the normal reach of the excavator's boom and bucket, which are positioned close to the worker.\", \"boxes\": [[155, 592, 191, 700]]}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": true, + "reason": "A worker is standing on the ground within the normal reach of the excavator's boom and bucket, which are positioned close to the worker.", + "boxes": [ + [ + 155, + 592, + 191, + 700 + ] + ] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 156, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002144" + ], + "batch_size": 1, + "generation_latency_seconds": 10.366267680074088, + "batch_wall_latency_seconds": 10.379591763019562, + "amortized_generation_ms_per_sample": 10366.267680074088, + "amortized_batch_wall_ms_per_sample": 10379.591763019562, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"Worker on the left is clearly bareheaded, missing a hard hat.\", \"boxes\": [[46, 383, 112, 545]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "Worker on the left is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 46, + 383, + 112, + 545 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 141, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001450" + ], + "batch_size": 1, + "generation_latency_seconds": 8.482930272002704, + "batch_wall_latency_seconds": 8.493541297968477, + "amortized_generation_ms_per_sample": 8482.930272002704, + "amortized_batch_wall_ms_per_sample": 8493.541297968477, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1683 + } + ] + }, + { + "sample_ids": [ + "0003429" + ], + "batch_size": 1, + "generation_latency_seconds": 10.942747003980912, + "batch_wall_latency_seconds": 11.023728625965305, + "amortized_generation_ms_per_sample": 10942.747003980912, + "amortized_batch_wall_ms_per_sample": 11023.728625965305, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker on the left is clearly bareheaded, missing a hard hat.\", \"boxes\": [[129, 331, 202, 555]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker on the left is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 129, + 331, + 202, + 555 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 143, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000205" + ], + "batch_size": 1, + "generation_latency_seconds": 12.731149753089994, + "batch_wall_latency_seconds": 12.753692528931424, + "amortized_generation_ms_per_sample": 12731.149753089994, + "amortized_batch_wall_ms_per_sample": 12753.692528931424, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The circular concrete structure has a deep opening in the center and the worker is standing near the edge without guardrails or fencing, indicating an unprotected edge over a deep drop.\", \"boxes\": [[0, 100, 1000, 1000]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The circular concrete structure has a deep opening in the center and the worker is standing near the edge without guardrails or fencing, indicating an unprotected edge over a deep drop.", + "boxes": [ + [ + 0, + 100, + 1000, + 1000 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 164, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004209" + ], + "batch_size": 1, + "generation_latency_seconds": 8.481906582019292, + "batch_wall_latency_seconds": 8.510338544961996, + "amortized_generation_ms_per_sample": 8481.906582019292, + "amortized_batch_wall_ms_per_sample": 8510.338544961996, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003623" + ], + "batch_size": 1, + "generation_latency_seconds": 11.558857796015218, + "batch_wall_latency_seconds": 11.585797557956539, + "amortized_generation_ms_per_sample": 11558.857796015218, + "amortized_batch_wall_ms_per_sample": 11585.79755795654, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": true, \"reason\": \"Worker on the upper level is exposed to a fall hazard without a safety harness or effective edge protection.\", \"boxes\": [[508, 31, 646, 152]]}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": true, + "reason": "Worker on the upper level is exposed to a fall hazard without a safety harness or effective edge protection.", + "boxes": [ + [ + 508, + 31, + 646, + 152 + ] + ] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 147, + "prompt_tokens": 1694 + } + ] + }, + { + "sample_ids": [ + "0002496" + ], + "batch_size": 1, + "generation_latency_seconds": 11.345404008985497, + "batch_wall_latency_seconds": 11.363981842063367, + "amortized_generation_ms_per_sample": 11345.404008985497, + "amortized_batch_wall_ms_per_sample": 11363.981842063367, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The top edge of the deep excavation pit lacks guardrails or fencing, exposing workers to a significant fall hazard.\", \"boxes\": [[42, 442, 998, 685]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The top edge of the deep excavation pit lacks guardrails or fencing, exposing workers to a significant fall hazard.", + "boxes": [ + [ + 42, + 442, + 998, + 685 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 149, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004049" + ], + "batch_size": 1, + "generation_latency_seconds": 8.281572339008562, + "batch_wall_latency_seconds": 8.321087789023295, + "amortized_generation_ms_per_sample": 8281.572339008562, + "amortized_batch_wall_ms_per_sample": 8321.087789023295, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001483" + ], + "batch_size": 1, + "generation_latency_seconds": 8.35940665891394, + "batch_wall_latency_seconds": 8.414654169930145, + "amortized_generation_ms_per_sample": 8359.40665891394, + "amortized_batch_wall_ms_per_sample": 8414.654169930145, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004197" + ], + "batch_size": 1, + "generation_latency_seconds": 11.471120469039306, + "batch_wall_latency_seconds": 11.495890544028953, + "amortized_generation_ms_per_sample": 11471.120469039306, + "amortized_batch_wall_ms_per_sample": 11495.890544028953, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": true, \"reason\": \"A worker is standing on the ground directly in the path of the excavator's bucket and within the machine's active swing zone.\", \"boxes\": [[638, 545, 668, 682]]}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": true, + "reason": "A worker is standing on the ground directly in the path of the excavator's bucket and within the machine's active swing zone.", + "boxes": [ + [ + 638, + 545, + 668, + 682 + ] + ] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 154, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003082" + ], + "batch_size": 1, + "generation_latency_seconds": 11.027026457944885, + "batch_wall_latency_seconds": 11.062949666986242, + "amortized_generation_ms_per_sample": 11027.026457944885, + "amortized_batch_wall_ms_per_sample": 11062.949666986242, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The top edge of the deep excavation on the left side lacks guardrails or fencing, exposing workers to a significant drop.\", \"boxes\": [[0, 0, 670, 640]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The top edge of the deep excavation on the left side lacks guardrails or fencing, exposing workers to a significant drop.", + "boxes": [ + [ + 0, + 0, + 670, + 640 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 148, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000758" + ], + "batch_size": 1, + "generation_latency_seconds": 11.055003611021675, + "batch_wall_latency_seconds": 11.07016703102272, + "amortized_generation_ms_per_sample": 11055.003611021675, + "amortized_batch_wall_ms_per_sample": 11070.16703102272, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": true, \"reason\": \"A worker is standing within the normal reach of the excavator's boom and bucket in the active work area.\", \"boxes\": [[636, 330, 725, 582]]}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": true, + "reason": "A worker is standing within the normal reach of the excavator's boom and bucket in the active work area.", + "boxes": [ + [ + 636, + 330, + 725, + 582 + ] + ] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 150, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000564" + ], + "batch_size": 1, + "generation_latency_seconds": 13.539180523017421, + "batch_wall_latency_seconds": 13.570679377065971, + "amortized_generation_ms_per_sample": 13539.180523017421, + "amortized_batch_wall_ms_per_sample": 13570.679377065971, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker on the right is walking on the construction site without a hard hat.\", \"boxes\": [[576, 515, 611, 632]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": true, \"reason\": \"A worker is standing very close to the excavator's tracks and rear counterweight, within its normal swing and travel zone.\", \"boxes\": [[472, 466, 508, 603]]}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker on the right is walking on the construction site without a hard hat.", + "boxes": [ + [ + 576, + 515, + 611, + 632 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": true, + "reason": "A worker is standing very close to the excavator's tracks and rear counterweight, within its normal swing and travel zone.", + "boxes": [ + [ + 472, + 466, + 508, + 603 + ] + ] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 187, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001580" + ], + "batch_size": 1, + "generation_latency_seconds": 8.109316511079669, + "batch_wall_latency_seconds": 8.123522659065202, + "amortized_generation_ms_per_sample": 8109.316511079669, + "amortized_batch_wall_ms_per_sample": 8123.522659065202, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1333 + } + ] + }, + { + "sample_ids": [ + "0002136" + ], + "batch_size": 1, + "generation_latency_seconds": 10.921932367025875, + "batch_wall_latency_seconds": 10.956237377016805, + "amortized_generation_ms_per_sample": 10921.932367025875, + "amortized_batch_wall_ms_per_sample": 10956.237377016805, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The upper edge of the deep excavation is accessible and lacks guardrails or fencing, exposing workers to a significant drop.\", \"boxes\": [[0, 62, 1000, 312]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The upper edge of the deep excavation is accessible and lacks guardrails or fencing, exposing workers to a significant drop.", + "boxes": [ + [ + 0, + 62, + 1000, + 312 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 149, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000949" + ], + "batch_size": 1, + "generation_latency_seconds": 10.729247406939976, + "batch_wall_latency_seconds": 10.767821826972067, + "amortized_generation_ms_per_sample": 10729.247406939976, + "amortized_batch_wall_ms_per_sample": 10767.821826972067, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"The driver inside the truck cab is clearly visible with a bare head, missing a hard hat.\", \"boxes\": [[378, 208, 408, 258]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "The driver inside the truck cab is clearly visible with a bare head, missing a hard hat.", + "boxes": [ + [ + 378, + 208, + 408, + 258 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 147, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003045" + ], + "batch_size": 1, + "generation_latency_seconds": 7.96182514692191, + "batch_wall_latency_seconds": 7.970436848932877, + "amortized_generation_ms_per_sample": 7961.82514692191, + "amortized_batch_wall_ms_per_sample": 7970.436848932877, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1129 + } + ] + }, + { + "sample_ids": [ + "0000013" + ], + "batch_size": 1, + "generation_latency_seconds": 8.21879173698835, + "batch_wall_latency_seconds": 8.24058064003475, + "amortized_generation_ms_per_sample": 8218.79173698835, + "amortized_batch_wall_ms_per_sample": 8240.58064003475, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003889" + ], + "batch_size": 1, + "generation_latency_seconds": 8.273510461091064, + "batch_wall_latency_seconds": 8.289232861017808, + "amortized_generation_ms_per_sample": 8273.510461091064, + "amortized_batch_wall_ms_per_sample": 8289.232861017808, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003261" + ], + "batch_size": 1, + "generation_latency_seconds": 8.405189790995792, + "batch_wall_latency_seconds": 8.4461084890645, + "amortized_generation_ms_per_sample": 8405.189790995792, + "amortized_batch_wall_ms_per_sample": 8446.1084890645, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001629" + ], + "batch_size": 1, + "generation_latency_seconds": 8.233813347993419, + "batch_wall_latency_seconds": 8.26478240697179, + "amortized_generation_ms_per_sample": 8233.813347993419, + "amortized_batch_wall_ms_per_sample": 8264.78240697179, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001569" + ], + "batch_size": 1, + "generation_latency_seconds": 8.24229718092829, + "batch_wall_latency_seconds": 8.27361940103583, + "amortized_generation_ms_per_sample": 8242.29718092829, + "amortized_batch_wall_ms_per_sample": 8273.61940103583, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002536" + ], + "batch_size": 1, + "generation_latency_seconds": 8.316832296084613, + "batch_wall_latency_seconds": 8.332275224965997, + "amortized_generation_ms_per_sample": 8316.832296084613, + "amortized_batch_wall_ms_per_sample": 8332.275224965997, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003053" + ], + "batch_size": 1, + "generation_latency_seconds": 8.387920169043355, + "batch_wall_latency_seconds": 8.417040045023896, + "amortized_generation_ms_per_sample": 8387.920169043355, + "amortized_batch_wall_ms_per_sample": 8417.040045023896, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002661" + ], + "batch_size": 1, + "generation_latency_seconds": 11.386560487910174, + "batch_wall_latency_seconds": 11.400971814990044, + "amortized_generation_ms_per_sample": 11386.560487910174, + "amortized_batch_wall_ms_per_sample": 11400.971814990044, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The top edge of the deep excavation trench lacks guardrails or fencing, exposing workers to a significant fall hazard.\", \"boxes\": [[0, 200, 780, 650]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The top edge of the deep excavation trench lacks guardrails or fencing, exposing workers to a significant fall hazard.", + "boxes": [ + [ + 0, + 200, + 780, + 650 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 150, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001664" + ], + "batch_size": 1, + "generation_latency_seconds": 8.633664096007124, + "batch_wall_latency_seconds": 8.660372731043026, + "amortized_generation_ms_per_sample": 8633.664096007124, + "amortized_batch_wall_ms_per_sample": 8660.372731043026, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001351" + ], + "batch_size": 1, + "generation_latency_seconds": 12.240043157013133, + "batch_wall_latency_seconds": 12.259637883980758, + "amortized_generation_ms_per_sample": 12240.043157013133, + "amortized_batch_wall_ms_per_sample": 12259.637883980758, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The top edge of the concrete retaining wall borders a deep excavation area and lacks guardrails or fencing, creating a fall hazard for workers on the upper level.\", \"boxes\": [[0, 500, 998, 580]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The top edge of the concrete retaining wall borders a deep excavation area and lacks guardrails or fencing, creating a fall hazard for workers on the upper level.", + "boxes": [ + [ + 0, + 500, + 998, + 580 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 159, + "prompt_tokens": 1683 + } + ] + }, + { + "sample_ids": [ + "0000304" + ], + "batch_size": 1, + "generation_latency_seconds": 8.286614268901758, + "batch_wall_latency_seconds": 8.305405738996342, + "amortized_generation_ms_per_sample": 8286.614268901758, + "amortized_batch_wall_ms_per_sample": 8305.405738996342, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1243 + } + ] + }, + { + "sample_ids": [ + "0004537" + ], + "batch_size": 1, + "generation_latency_seconds": 10.83758004498668, + "batch_wall_latency_seconds": 10.849978967919014, + "amortized_generation_ms_per_sample": 10837.58004498668, + "amortized_batch_wall_ms_per_sample": 10849.978967919014, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": true, \"reason\": \"Worker is standing on rebar at a significant height without a safety harness, exposed to a fall.\", \"boxes\": [[538, 292, 706, 568]]}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": true, + "reason": "Worker is standing on rebar at a significant height without a safety harness, exposed to a fall.", + "boxes": [ + [ + 538, + 292, + 706, + 568 + ] + ] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 148, + "prompt_tokens": 1405 + } + ] + }, + { + "sample_ids": [ + "0003801" + ], + "batch_size": 1, + "generation_latency_seconds": 8.35220990900416, + "batch_wall_latency_seconds": 8.370414639008231, + "amortized_generation_ms_per_sample": 8352.20990900416, + "amortized_batch_wall_ms_per_sample": 8370.414639008231, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004321" + ], + "batch_size": 1, + "generation_latency_seconds": 11.045618020929396, + "batch_wall_latency_seconds": 11.056004835991189, + "amortized_generation_ms_per_sample": 11045.618020929396, + "amortized_batch_wall_ms_per_sample": 11056.004835991189, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker on foot near the concrete pump hose is clearly bareheaded, missing a hard hat.\", \"boxes\": [[451, 686, 468, 758]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker on foot near the concrete pump hose is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 451, + 686, + 468, + 758 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 147, + "prompt_tokens": 1393 + } + ] + }, + { + "sample_ids": [ + "0001878" + ], + "batch_size": 1, + "generation_latency_seconds": 12.032795147038996, + "batch_wall_latency_seconds": 12.044023220078088, + "amortized_generation_ms_per_sample": 12032.795147038996, + "amortized_batch_wall_ms_per_sample": 12044.023220078088, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": true, \"reason\": \"A worker is standing on the top of the concrete pump truck chassis, which is elevated above the ground, without any visible safety harness or fall protection equipment.\", \"boxes\": [[483, 526, 518, 625]]}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": true, + "reason": "A worker is standing on the top of the concrete pump truck chassis, which is elevated above the ground, without any visible safety harness or fall protection equipment.", + "boxes": [ + [ + 483, + 526, + 518, + 625 + ] + ] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 159, + "prompt_tokens": 1705 + } + ] + } + ], + "mean_generation_ms_per_sample": 9800.482400676628, + "median_generation_ms_per_sample": 8558.297184004914, + "mean_batch_wall_ms_per_sample": 9822.475872695577, + "median_batch_wall_ms_per_sample": 8585.355638002511, + "mean_prompt_tokens": 1624.5625, + "mean_completion_tokens": 130.796875, + "latency_scope": "A800 vLLM-0.18.0 native generation latency with the frozen task prompt and structured-output schema; model loading and warmup excluded. Batch wall time includes the real request/image preparation path.", + "flops_scope": "This file measures latency only. Counted FLOPs are measured separately by the HF reference probe and are not claimed to be complete vLLM kernel FLOPs." +} \ No newline at end of file diff --git a/paper_results/data/main_table_cost_qwen_20260912/components/yolo26x_router.json b/paper_results/data/main_table_cost_qwen_20260912/components/yolo26x_router.json new file mode 100644 index 0000000000000000000000000000000000000000..cef08acee599ed6dc1e24e3c873932ba815f9e87 --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/components/yolo26x_router.json @@ -0,0 +1,238 @@ +{ + "status": "complete", + "run": "shared_12_yolo26x_Qwen3.8", + "config": { + "fraction": 1.0, + "seed": 42, + "freeze_encoder": true, + "fusion": "concat", + "harm_cost": 1.0, + "batch_size": 256, + "weight_decay": 0.01, + "max_epochs": 50, + "patience": 8, + "cpu_threads": 1, + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90, + "expert": "yolo26x", + "task": "construction", + "endpoint": "Qwen3.8", + "branches": [ + "image", + "hidden", + "confidence", + "output" + ], + "hidden_layer": "feature_blocks__layer_16", + "id": "shared_12_yolo26x_Qwen3.8", + "stage": "shared_mandatory_inputs" + }, + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_shared_image_hidden_20260910/runs/shared_12_yolo26x_Qwen3.8/best.pt", + "score": "logit_rescue-logit_harm", + "test_rows_recomputed": 3004, + "selected_decisions_match_main": true, + "max_score_abs_difference": 0.0, + "actual_call_fraction": 0.4876830892143808, + "sample_count": 64, + "sample_ids": [ + "0002628", + "0003242", + "0001568", + "0001119", + "0003648", + "0000187", + "0001891", + "0001689", + "0003372", + "0002549", + "0000915", + "0003633", + "0004424", + "0001627", + "0001562", + "0002018", + "0000254", + "0002719", + "0002730", + "0004413", + "0000891", + "0000150", + "0003223", + "0004731", + "0001587", + "0000483", + "0001116", + "0003376", + "0001225", + "0003082", + "0004447", + "0000283", + "0000563", + "0001254", + "0004759", + "0001868", + "0001860", + "0004087", + "0002962", + "0001802", + "0002973", + "0002648", + "0004350", + "0004377", + "0004809", + "0003694", + "0001572", + "0001550", + "0000213", + "0002266", + "0002445", + "0004513", + "0002124", + "0004391", + "0003679", + "0002902", + "0002508", + "0004767", + "0003840", + "0003498", + "0001579", + "0001748", + "0004706", + "0004387" + ], + "batch_size": 1, + "latency_ms": [ + 17.35539303626865, + 13.579263002611697, + 15.319477999582887, + 14.879688038490713, + 24.25052202306688, + 20.667891018092632, + 20.015343907289207, + 19.221558002755046, + 20.937222987413406, + 26.26270893961191, + 17.899635946378112, + 16.57471200451255, + 18.844021018594503, + 21.3212959934026, + 19.545868039131165, + 64.57463896367699, + 20.962884067557752, + 24.004184058867395, + 20.19065897911787, + 18.50342599209398, + 19.576229038648307, + 25.253725005313754, + 21.430211956612766, + 25.169604923576117, + 17.8982320940122, + 21.415626979433, + 22.9903319850564, + 229.59673800505698, + 19.89229000173509, + 18.98771501146257, + 18.157949903979897, + 22.423187969252467, + 19.864327972754836, + 21.502194926142693, + 18.544791964814067, + 50.5554600385949, + 20.046600024215877, + 16.299219918437302, + 18.668118049390614, + 24.017272982746363, + 24.73958197515458, + 18.68593308608979, + 20.250934991054237, + 23.676551994867623, + 23.003229987807572, + 24.6098650386557, + 21.308812079951167, + 17.95686304103583, + 18.23423698078841, + 14.343310962431133, + 25.8766250917688, + 18.20633898023516, + 30.77617404051125, + 19.3530690157786, + 15.836970997042954, + 19.045730936340988, + 16.162591986358166, + 25.458221905864775, + 18.277363968081772, + 52.543785073794425, + 22.9536690749228, + 17.81571004539728, + 18.550690030679107, + 21.854626946151257 + ], + "mean_latency_ms_per_input": 25.261267359383055, + "head_mean_latency_ms": 0.486134999897331, + "gflops_per_input": [ + 3.183754752, + 3.183754752, + 3.183754752, + 3.183754752, + 3.183754752, + 3.183754752, + 3.183754752, + 3.183754752 + ], + "mean_gflops_per_input": 3.183754752, + "operators": [ + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + } + ], + "scope": "image loading, bilinear resize, tokenize when used, two image slots (including gray placeholder), active distilled encoders, feature assembly and original routing head; cached specialist features; supported FLOPs only" +} \ No newline at end of file diff --git a/paper_results/data/main_table_cost_qwen_20260912/components/yolo26x_specialist.json b/paper_results/data/main_table_cost_qwen_20260912/components/yolo26x_specialist.json new file mode 100644 index 0000000000000000000000000000000000000000..251decaacc2a6eb1affbe81ae0c03c9f13b031aa --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/components/yolo26x_specialist.json @@ -0,0 +1,254 @@ +{ + "component": "specialist", + "expert": "yolo26x", + "task": "construction", + "device": "cuda:0", + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/models/construction/yolo26x.pt", + "sample_count": 64, + "batch_size": 1, + "sample_ids": [ + "0000944", + "0003511", + "0003365", + "0001351", + "0002149", + "0003368", + "0000176", + "0004487", + "0000013", + "0003583", + "0000758", + "0001291", + "0001451", + "0002755", + "0003088", + "0004538", + "0004324", + "0000516", + "0003893", + "0003475", + "0002137", + "0003272", + "0001711", + "0003262", + "0001470", + "0002145", + "0002662", + "0000756", + "0001631", + "0001924", + "0002496", + "0001443", + "0004053", + "0002168", + "0001881", + "0001666", + "0000304", + "0001252", + "0004844", + "0000949", + "0003430", + "0003807", + "0004199", + "0002538", + "0002091", + "0001569", + "0003048", + "0002854", + "0000380", + "0001234", + "0001484", + "0003180", + "0000824", + "0003784", + "0000894", + "0004168", + "0000564", + "0004296", + "0003054", + "0001583", + "0003386", + "0000205", + "0004214", + "0003625" + ], + "model_load_seconds_excluded_from_latency": true, + "forward_latency_ms_per_sample": [ + 41.741790771484375, + 51.50377655029297, + 78.82457733154297, + 87.29910278320312, + 49.47385787963867, + 49.40816116333008, + 168.38259887695312, + 42.57542419433594, + 40.342529296875, + 54.95475387573242, + 48.86323165893555, + 40.684993743896484, + 282.5484619140625, + 47.938270568847656, + 45.019649505615234, + 268.04327392578125, + 51.39228820800781, + 41.36457443237305, + 54.01919937133789, + 43.2599983215332, + 47.13923263549805, + 41.1484489440918, + 35.99542236328125, + 80.46995544433594, + 83.43251037597656, + 48.88550567626953, + 171.4989776611328, + 58.917728424072266, + 46.90486526489258, + 56.416385650634766, + 44.49577713012695, + 49.42009735107422, + 43.6354866027832, + 55.22761535644531, + 48.02000045776367, + 39.44412612915039, + 30.26905632019043, + 492.74456787109375, + 45.74156951904297, + 39.223392486572266, + 50.18822479248047, + 43.890113830566406, + 51.966495513916016, + 47.814815521240234, + 54.13961410522461, + 39.215328216552734, + 56.004703521728516, + 39.528831481933594, + 396.5307312011719, + 83.4832992553711, + 50.0503044128418, + 153.1714630126953, + 61.07219314575195, + 52.57244873046875, + 42.75983810424805, + 38.4617919921875, + 44.08700942993164, + 53.36006546020508, + 45.519649505615234, + 290.8823547363281, + 48.10774230957031, + 53.34476852416992, + 51.016319274902344, + 42.9719352722168 + ], + "end_to_end_latency_ms_per_sample": [ + 35.17372906208038, + 48.07086195796728, + 46.07532289810479, + 45.34715902991593, + 46.541396994143724, + 45.594630064442754, + 45.38595397025347, + 39.57494895439595, + 39.812199072912335, + 51.36631103232503, + 42.81833197455853, + 33.49219704978168, + 37.3103239107877, + 38.96563302259892, + 40.327758993953466, + 44.32409000582993, + 43.25311293359846, + 33.99710892699659, + 49.12354901898652, + 45.043769059702754, + 47.554407035931945, + 37.903546006418765, + 32.56980108562857, + 35.192839917726815, + 39.34567596297711, + 46.67255701497197, + 49.70169102307409, + 52.48237494379282, + 43.415413005277514, + 48.36420400533825, + 48.96194895263761, + 51.6092429170385, + 37.311508087441325, + 50.858690054155886, + 43.94579504150897, + 36.09434806276113, + 26.66287892498076, + 41.9749760767445, + 34.900374012067914, + 39.04577391222119, + 51.60050210542977, + 46.00996105000377, + 53.40714694466442, + 51.78673507180065, + 54.96632098220289, + 46.47863702848554, + 47.48446703888476, + 36.593522992916405, + 37.32479899190366, + 26.43477905075997, + 39.97986298054457, + 55.87899999227375, + 40.821335976943374, + 45.88699492160231, + 40.707007981836796, + 34.627774031832814, + 43.4188200160861, + 49.56604307517409, + 39.82124803587794, + 249.81214595027268, + 38.2277499884367, + 48.29735495150089, + 47.6501100929454, + 40.76927900314331 + ], + "mean_forward_latency_ms_per_sample": 78.70017620921135, + "median_forward_latency_ms_per_sample": 49.41412925720215, + "mean_end_to_end_latency_ms_per_sample": 46.30809425361804, + "median_end_to_end_latency_ms_per_sample": 43.68230752879754, + "flop_profiles": [ + { + "batch_size": 1, + "flops_counted": 116518072320, + "gflops_counted_per_sample": 116.51807232, + "operators": { + "aten.convolution": 116319006720, + "aten.bmm": 199065600 + } + }, + { + "batch_size": 1, + "flops_counted": 145709798400, + "gflops_counted_per_sample": 145.7097984, + "operators": { + "aten.convolution": 145398758400, + "aten.bmm": 311040000 + } + }, + { + "batch_size": 1, + "flops_counted": 145709798400, + "gflops_counted_per_sample": 145.7097984, + "operators": { + "aten.convolution": 145398758400, + "aten.bmm": 311040000 + } + }, + { + "batch_size": 1, + "flops_counted": 135976458240, + "gflops_counted_per_sample": 135.97645824, + "operators": { + "aten.convolution": 135705507840, + "aten.bmm": 270950400 + } + } + ], + "mean_counted_gflops_per_sample": 135.97853184000002, + "flops_scope": "GPU FlopCounterMode around the real Ultralytics predict path; multiply-add counted as 2 FLOPs. Elementwise, postprocessing and unsupported custom ops may be missing.", + "latency_scope": "A800 wall-clock latency for real Ultralytics prediction including image loading/preprocess, forward and result postprocessing, excluding model load; batch processing uses the recorded batch size.", + "completed": true +} \ No newline at end of file diff --git a/paper_results/data/main_table_cost_qwen_20260912/cost_summary.json b/paper_results/data/main_table_cost_qwen_20260912/cost_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..dff4f5c3e165c1e5a0fdf467d55e836ed5c10706 --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/cost_summary.json @@ -0,0 +1,75 @@ +{ + "components": { + "grefcoco|instancevg|Qwen3.8": { + "specialist_gflops": 7.502680064, + "router_gflops": 3.642171392, + "large_gflops": 32554.061985335, + "specialist_ms": 46.08042127620138, + "router_ms": 17.032772149832454, + "large_ms": 4071.4221567668574, + "large_samples": 64, + "flops_probes": 8, + "specialist_samples": 64, + "router_samples": 64, + "batch_size": 1 + }, + "construction|yolo26x|Qwen3.8": { + "specialist_gflops": 135.97853184000002, + "router_gflops": 3.183754752, + "large_gflops": 96874.712551641, + "specialist_ms": 46.30809425361804, + "router_ms": 25.261267359383055, + "large_ms": 9822.475872695577, + "large_samples": 64, + "flops_probes": 8, + "specialist_samples": 64, + "router_samples": 64, + "batch_size": 1 + } + }, + "selected_points": [ + { + "pair": "grefcoco|instancevg|Qwen3.8", + "method": "learned", + "nominal_validation_budget": 0.56, + "actual_test_call_fraction": 0.5355755324277571, + "test_metric": 0.7528287440093016, + "estimated_gflops_per_input": 17446.303931938, + "estimated_time_ms_per_input": 2243.6672827746106 + }, + { + "pair": "grefcoco|instancevg|Qwen3.8", + "method": "confidence", + "nominal_validation_budget": 0.71, + "actual_test_call_fraction": 0.6838612710206166, + "test_metric": 0.7243286163967898, + "estimated_gflops_per_input": 22269.96488623913, + "estimated_time_ms_per_input": 2830.3683522642846 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "nominal_validation_budget": 0.5, + "actual_test_call_fraction": 0.4876830892143808, + "test_metric": 0.29875800068449215, + "estimated_gflops_per_input": 47383.321370531434, + "estimated_time_ms_per_input": 4861.824738942901 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "nominal_validation_budget": 0.93, + "actual_test_call_fraction": 0.9221038615179761, + "test_metric": 0.2597490004762105, + "estimated_gflops_per_input": 89464.5250591521, + "estimated_time_ms_per_input": 9103.651026133362 + } + ], + "protocol": { + "method": "original main-table logit contrast; original checkpoints and validation thresholds; seed 42", + "large_timing": "64 measured examples; batch size one; same full task prompt; two separate warmup examples", + "cost": "serial component estimates, not a timed integrated service; includes active distilled encoders and routing head only for learned", + "flops": "multiply-add=2; supported operators; mean prefill (including visual embedding) plus average generated length times one-token decode probe", + "scope_limits": "mean call cost extrapolated across policies; no policy-specific input/output-length costs; specialist-feature extraction overhead is not separately measured; confidence threshold overhead not separately timed" + } +} \ No newline at end of file diff --git a/paper_results/data/main_table_cost_qwen_20260912/fixed_budget_cost_performance.csv b/paper_results/data/main_table_cost_qwen_20260912/fixed_budget_cost_performance.csv new file mode 100644 index 0000000000000000000000000000000000000000..f4e38dc91cc3bc9b3bfe25187fb66aea99cd8d67 --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/fixed_budget_cost_performance.csv @@ -0,0 +1,29 @@ +pair,method,nominal_validation_budget,actual_test_call_fraction,test_metric,estimated_gflops_per_input,estimated_time_ms_per_input +grefcoco|instancevg|Qwen3.8,learned,0.0,0.0,0.6695119530385957,11.144851456,63.11319342603383 +grefcoco|instancevg|Qwen3.8,learned,0.05,0.11998411933187761,0.7164449990074583,3917.1153094417755,551.619195333999 +grefcoco|instancevg|Qwen3.8,learned,0.1,0.16822164875365114,0.7290077418257097,5487.442832257611,748.0145414095009 +grefcoco|instancevg|Qwen3.8,learned,0.25,0.30176105266142983,0.748092901908516,9834.692864556126,1291.7098292810697 +grefcoco|instancevg|Qwen3.8,learned,0.5,0.487451436349715,0.7535093440716899,15879.6691252252,2047.733771728093 +grefcoco|instancevg|Qwen3.8,learned,0.75,0.6870657629810283,0.7512973938689278,22377.92628754188,2860.4479639829183 +grefcoco|instancevg|Qwen3.8,learned,1.0,1.0,0.7447182599325072,32565.206836791,4134.535350192891 +grefcoco|instancevg|Qwen3.8,confidence,0.0,0.0,0.6695119530385957,7.502680064,46.08042127620138 +grefcoco|instancevg|Qwen3.8,confidence,0.05,0.012534384482318578,0.6666194027734452,415.54780944942013,97.11319197894791 +grefcoco|instancevg|Qwen3.8,confidence,0.1,0.0345971698380739,0.6647761109378102,1133.7810914898203,186.94010511636148 +grefcoco|instancevg|Qwen3.8,confidence,0.25,0.12477667810452883,0.6594731021183677,4069.490393403024,554.0989531587461 +grefcoco|instancevg|Qwen3.8,confidence,0.5,0.2345234381646485,0.662762669086578,7642.1932230898465,1000.9243437008932 +grefcoco|instancevg|Qwen3.8,confidence,0.75,0.7478093185491875,0.7311629753566061,24351.733589325377,3090.727849854088 +grefcoco|instancevg|Qwen3.8,confidence,1.0,1.0,0.7447182599325072,32561.564665399,4117.502578043059 +construction|yolo26x|Qwen3.8,learned,0.0,0.0,0.18129624718318377,139.16228659200002,71.56936161300109 +construction|yolo26x|Qwen3.8,learned,0.05,0.045938748335552594,0.19531128485766117,4589.465326580834,522.8016087608005 +construction|yolo26x|Qwen3.8,learned,0.1,0.09920106524633822,0.20211074868345033,9749.236967147599,1045.9694315408578 +construction|yolo26x|Qwen3.8,learned,0.25,0.25432756324900135,0.2504846177578749,24777.071870298303,2569.695715387775 +construction|yolo26x|Qwen3.8,learned,0.5,0.4876830892143808,0.29875800068449215,47383.321370531434,4861.824738942901 +construction|yolo26x|Qwen3.8,learned,0.75,0.7709720372836218,0.28730364578560735,74826.85678379593,7644.4235963543315 +construction|yolo26x|Qwen3.8,learned,1.0,1.0,0.27001009067886245,97013.87483823301,9894.045234308578 +construction|yolo26x|Qwen3.8,confidence,0.0,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.05,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.1,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.25,0.24800266311584554,0.18026802885144627,24161.165233228996,2482.3082690732604 +construction|yolo26x|Qwen3.8,confidence,0.5,0.5013315579227696,0.18015785955296285,48702.32909867467,4970.625226170908 +construction|yolo26x|Qwen3.8,confidence,0.75,0.7486684420772304,0.22103445955201576,72663.01865455658,7400.0858032057995 +construction|yolo26x|Qwen3.8,confidence,1.0,1.0,0.27001009067886245,97010.691083481,9868.783966949195 diff --git a/paper_results/data/main_table_cost_qwen_20260912/full_cost_performance_curves.csv b/paper_results/data/main_table_cost_qwen_20260912/full_cost_performance_curves.csv new file mode 100644 index 0000000000000000000000000000000000000000..e25f1dec04edc9639248485b8cfa9becbfeb6fd6 --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/full_cost_performance_curves.csv @@ -0,0 +1,405 @@ +pair,method,nominal_validation_budget,actual_test_call_fraction,test_metric,estimated_gflops_per_input,estimated_time_ms_per_input +grefcoco|instancevg|Qwen3.8,learned,0.0,0.0,0.6695119530385957,11.144851456,63.11319342603383 +grefcoco|instancevg|Qwen3.8,learned,0.01,0.04701812097666109,0.6878314380512152,1541.775676164205,254.54381293995635 +grefcoco|instancevg|Qwen3.8,learned,0.02,0.06706746448118424,0.6980404389870403,2194.4632473757256,336.17315431290154 +grefcoco|instancevg|Qwen3.8,learned,0.03,0.08720188299350594,0.7065195814309616,2849.9203556645225,418.1488719575849 +grefcoco|instancevg|Qwen3.8,learned,0.04,0.1055780846779911,0.712389756969061,3448.14036435627,492.9661466530142 +grefcoco|instancevg|Qwen3.8,learned,0.05,0.11998411933187761,0.7164449990074583,3917.1153094417755,551.619195333999 +grefcoco|instancevg|Qwen3.8,learned,0.06,0.13243342880639764,0.7199047159912656,4322.390901749918,602.3055897650073 +grefcoco|instancevg|Qwen3.8,learned,0.07,0.14235884638289426,0.7222017412018262,4645.503560765523,642.7161548011189 +grefcoco|instancevg|Qwen3.8,learned,0.08,0.1511215721861441,0.7239032413577972,4930.765879725012,678.392910790143 +grefcoco|instancevg|Qwen3.8,learned,0.09,0.15957235629413266,0.7263136999120892,5205.873229401156,712.7996204494608 +grefcoco|instancevg|Qwen3.8,learned,0.1,0.16822164875365114,0.7290077418257097,5487.442832257611,748.0145414095009 +grefcoco|instancevg|Qwen3.8,learned,0.11,0.17695601622096815,0.7316450670674645,5771.781972191343,783.575838641279 +grefcoco|instancevg|Qwen3.8,learned,0.12,0.18855457561750277,0.7345943340044806,6149.362193726721,830.7984703549065 +grefcoco|instancevg|Qwen3.8,learned,0.13,0.19751580977228256,0.7366361341916456,6441.086765866523,867.2834376446528 +grefcoco|instancevg|Qwen3.8,learned,0.14,0.20664719394265946,0.7385361426991465,6738.350412160879,904.4611574778754 +grefcoco|instancevg|Qwen3.8,learned,0.15,0.2172532115815444,0.7402376428551173,7083.619367794696,947.6427326878917 +grefcoco|instancevg|Qwen3.8,learned,0.16,0.22717862915804102,0.7420242180188866,7406.7320268103,988.0532977240033 +grefcoco|instancevg|Qwen3.8,learned,0.17,0.2361398633128208,0.7436122848311261,7698.4565989501025,1024.5382650137497 +grefcoco|instancevg|Qwen3.8,learned,0.18,0.24362646399909255,0.7448033349403057,7942.175861750445,1055.0193769267023 +grefcoco|instancevg|Qwen3.8,learned,0.19,0.2507727646541701,0.7456540850182911,8174.816976241679,1084.1149837527025 +grefcoco|instancevg|Qwen3.8,learned,0.2,0.25942205711368854,0.7459943850494853,8456.386579098133,1119.3299047127425 +grefcoco|instancevg|Qwen3.8,learned,0.21,0.2690922496667896,0.746419760088478,8771.19062688191,1158.7013409336398 +grefcoco|instancevg|Qwen3.8,learned,0.22,0.2781385588293679,0.7467600601196722,9065.684736098987,1195.5326844951244 +grefcoco|instancevg|Qwen3.8,learned,0.23,0.2868445679607521,0.7473555851742619,9349.100697006961,1230.9785229696565 +grefcoco|instancevg|Qwen3.8,learned,0.24,0.29430281031109096,0.7480645435725831,9591.896780781542,1261.344176125363 +grefcoco|instancevg|Qwen3.8,learned,0.25,0.30176105266142983,0.748092901908516,9834.692864556126,1291.7098292810697 +grefcoco|instancevg|Qwen3.8,learned,0.26,0.30995661174602274,0.7486033519553073,10101.491603000439,1325.0774101251732 +grefcoco|instancevg|Qwen3.8,learned,0.27,0.3175850041119587,0.7494541020332927,10349.826760929574,1356.135815824356 +grefcoco|instancevg|Qwen3.8,learned,0.28,0.3262342965714772,0.7501630604316138,10631.396363786029,1391.350736784396 +grefcoco|instancevg|Qwen3.8,learned,0.29,0.3324731304767036,0.7505600771346738,10834.49574945298,1416.7516633785233 +grefcoco|instancevg|Qwen3.8,learned,0.3,0.3399313728270425,0.7509570938377336,11077.291833227562,1447.11731653423 +grefcoco|instancevg|Qwen3.8,learned,0.31,0.34824036525536683,0.7511839605251964,11347.783287774913,1480.9467324073175 +grefcoco|instancevg|Qwen3.8,learned,0.32,0.3557553242775714,0.751269035532995,11592.425729601013,1511.5433030775164 +grefcoco|instancevg|Qwen3.8,learned,0.33,0.3627598332529847,0.7511272438533307,11820.450948963453,1540.0616161172863 +grefcoco|instancevg|Qwen3.8,learned,0.34,0.37013300059552506,0.7512123188611293,12060.47749566076,1570.080893001255 +grefcoco|instancevg|Qwen3.8,learned,0.35,0.3783569180160508,0.751439185548592,12328.199413130831,1603.5639326026044 +grefcoco|instancevg|Qwen3.8,learned,0.36,0.3854748603351955,0.7517511272438533,12559.917348596306,1632.5440806713586 +grefcoco|instancevg|Qwen3.8,learned,0.37,0.39279131100587017,0.752176502282846,12798.097537242094,1662.3324400408353 +grefcoco|instancevg|Qwen3.8,learned,0.38,0.40013612001247767,0.7522332189547117,13037.200904913643,1692.2362581675577 +grefcoco|instancevg|Qwen3.8,learned,0.39,0.406573462269234,0.7525735189859059,13246.762543760904,1718.445396062407 +grefcoco|instancevg|Qwen3.8,learned,0.4,0.4153929047443496,0.7528003856733687,13533.871220771913,1754.3530695659233 +grefcoco|instancevg|Qwen3.8,learned,0.41,0.42151830530584466,0.75311232736863,13733.27789033583,1779.2921611310667 +grefcoco|instancevg|Qwen3.8,learned,0.42,0.42920341434364634,0.7535093440716899,13983.459406316484,1810.5814843447415 +grefcoco|instancevg|Qwen3.8,learned,0.43,0.4367750900377166,0.7537362107591526,14229.948206194102,1841.4089725294323 +grefcoco|instancevg|Qwen3.8,learned,0.44,0.4450273657941752,0.753679494087287,14498.593302689931,1875.0074708880277 +grefcoco|instancevg|Qwen3.8,learned,0.45,0.45143634971499874,0.7537929274310183,14707.231762511437,1901.1011500256313 +grefcoco|instancevg|Qwen3.8,learned,0.46,0.4578736919717551,0.7536227774154213,14916.7934013587,1927.3102879204807 +grefcoco|instancevg|Qwen3.8,learned,0.47,0.4659274593766838,0.7537929274310183,15178.976244674219,1960.100574978354 +grefcoco|instancevg|Qwen3.8,learned,0.48,0.4741797351331424,0.7538780024388169,15447.621341170048,1993.6990733369496 +grefcoco|instancevg|Qwen3.8,learned,0.49,0.4806454357258316,0.7537362107591526,15658.106159043073,2020.0236699890452 +grefcoco|instancevg|Qwen3.8,learned,0.5,0.487451436349715,0.7535093440716899,15879.6691252252,2047.733771728093 +grefcoco|instancevg|Qwen3.8,learned,0.51,0.49482460369225534,0.7537645690950855,16119.695671922505,2077.7530486120613 +grefcoco|instancevg|Qwen3.8,learned,0.52,0.501573887644273,0.7535660607435556,16339.412280053117,2105.232232836617 +grefcoco|instancevg|Qwen3.8,learned,0.53,0.5105351217990528,0.7532257607123614,16631.136852192918,2141.7172001263634 +grefcoco|instancevg|Qwen3.8,learned,0.54,0.519241130930437,0.7530556106967643,16914.552813100894,2177.163038600896 +grefcoco|instancevg|Qwen3.8,learned,0.55,0.5268128066245072,0.7528571023452344,17161.041612978508,2207.9905267855866 +grefcoco|instancevg|Qwen3.8,learned,0.56,0.5355755324277571,0.7528287440093016,17446.303931938,2243.6672827746106 +grefcoco|instancevg|Qwen3.8,learned,0.57,0.5419561580126478,0.753282477384227,17654.019212733747,2269.645503154968 +grefcoco|instancevg|Qwen3.8,learned,0.58,0.5495845503785838,0.7528287440093016,17902.35437066288,2300.703908854151 +grefcoco|instancevg|Qwen3.8,learned,0.59,0.5562771176587358,0.7527153106655702,18120.224620741974,2327.952175564215 +grefcoco|instancevg|Qwen3.8,learned,0.6,0.564047301704336,0.7526585939937045,18373.175673799906,2359.587875049628 +grefcoco|instancevg|Qwen3.8,learned,0.61,0.5725831608201231,0.7528287440093016,18651.052560553326,2394.3409609806836 +grefcoco|instancevg|Qwen3.8,learned,0.62,0.5818846950060971,0.7528571023452344,18953.855281002234,2432.2114333573827 +grefcoco|instancevg|Qwen3.8,learned,0.63,0.5903638374500184,0.7528571023452344,19229.885809704134,2466.7336017739462 +grefcoco|instancevg|Qwen3.8,learned,0.64,0.5973683464254318,0.7526869523296373,19457.911029066578,2495.2519148137167 +grefcoco|instancevg|Qwen3.8,learned,0.65,0.6042594220571137,0.7526018773218388,19682.24353232598,2523.3083928245023 +grefcoco|instancevg|Qwen3.8,learned,0.66,0.6122848311261095,0.752545160649973,19943.503196615744,2555.9832211251296 +grefcoco|instancevg|Qwen3.8,learned,0.67,0.6198848651561126,0.7522048606187789,20190.91517551912,2586.9261680670666 +grefcoco|instancevg|Qwen3.8,learned,0.68,0.6280520659047727,0.752176502282846,20456.790734937673,2620.178290153924 +grefcoco|instancevg|Qwen3.8,learned,0.69,0.6363894166690298,0.7522048606187789,20728.20536851078,2654.1231647842574 +grefcoco|instancevg|Qwen3.8,learned,0.7,0.6444999007458242,0.7521197856109804,20992.234569877815,2687.144369356623 +grefcoco|instancevg|Qwen3.8,learned,0.71,0.6527521765022829,0.7519212772594505,21260.87966637365,2720.742867715219 +grefcoco|instancevg|Qwen3.8,learned,0.72,0.6601253438448231,0.7518362022516518,21500.906213070954,2750.7621445991867 +grefcoco|instancevg|Qwen3.8,learned,0.73,0.6697955363979242,0.7518362022516518,21815.71026085473,2790.1335808200847 +grefcoco|instancevg|Qwen3.8,learned,0.74,0.6775373621075915,0.7518645605875848,22067.7381348869,2821.6538215482515 +grefcoco|instancevg|Qwen3.8,learned,0.75,0.6870657629810283,0.7512973938689278,22377.92628754188,2860.4479639829183 +grefcoco|instancevg|Qwen3.8,learned,0.76,0.6956299804327482,0.7514108272126592,22656.726353321057,2895.31650867122 +grefcoco|instancevg|Qwen3.8,learned,0.77,0.7044494229078637,0.7514959022204577,22943.835030332066,2931.224182174736 +grefcoco|instancevg|Qwen3.8,learned,0.78,0.7129852820236509,0.7510705271814649,23221.711917085486,2965.9772681057925 +grefcoco|instancevg|Qwen3.8,learned,0.79,0.7243853330686555,0.7506735104784051,23592.82988544055,3012.3916885186977 +grefcoco|instancevg|Qwen3.8,learned,0.8,0.7337152255905623,0.7507302271502708,23896.555784915217,3050.3776196526424 +grefcoco|instancevg|Qwen3.8,learned,0.81,0.744377959901313,0.7505033604628081,24243.671098600553,3093.7901123771508 +grefcoco|instancevg|Qwen3.8,learned,0.82,0.7522048606187789,0.7503048521112781,24498.46850971,3125.656729377056 +grefcoco|instancevg|Qwen3.8,learned,0.83,0.7607123613986331,0.7503332104472109,24775.42221743766,3160.2943565508654 +grefcoco|instancevg|Qwen3.8,learned,0.84,0.770779570654794,0.749964552080084,25103.15077158206,3201.2822153732072 +grefcoco|instancevg|Qwen3.8,learned,0.85,0.7827467884184556,0.749964552080084,25492.7323204523,3250.005810931033 +grefcoco|instancevg|Qwen3.8,learned,0.86,0.793466239401072,0.7498511187363526,25841.693992189157,3293.649221170034 +grefcoco|instancevg|Qwen3.8,learned,0.87,0.8059439072115249,0.7499361937441511,26247.89276352306,3344.451074358288 +grefcoco|instancevg|Qwen3.8,learned,0.88,0.8164932081785441,0.7497093270566884,26591.315361105357,3387.4017320538123 +grefcoco|instancevg|Qwen3.8,learned,0.89,0.8270425091455633,0.7494257436973598,26934.737958687656,3430.3523897493365 +grefcoco|instancevg|Qwen3.8,learned,0.9,0.8367410600345971,0.749397385361427,27250.46518549719,3469.83928472748 +grefcoco|instancevg|Qwen3.8,learned,0.91,0.8491903695091172,0.74883021864277,27655.740777805335,3520.5256791584884 +grefcoco|instancevg|Qwen3.8,learned,0.92,0.8618098289992343,0.7487167852990386,28066.55544426803,3571.904826132973 +grefcoco|instancevg|Qwen3.8,learned,0.93,0.8745710801690156,0.7486884269631058,28481.98600585952,3623.8612668936876 +grefcoco|instancevg|Qwen3.8,learned,0.94,0.8879845730652526,0.7486884269631058,28918.649685043467,3678.4732590710614 +grefcoco|instancevg|Qwen3.8,learned,0.95,0.9000085075007799,0.7484332019397102,29310.07759196523,3727.4277721433796 +grefcoco|instancevg|Qwen3.8,learned,0.96,0.9125996086549641,0.7477809602132547,29719.969079402166,3778.691460360618 +grefcoco|instancevg|Qwen3.8,learned,0.97,0.9269489266369849,0.7476391685335905,30187.09766643615,3837.11359152711 +grefcoco|instancevg|Qwen3.8,learned,0.98,0.9459773700479256,0.7472988685023962,30806.550792720354,3914.5864176391988 +grefcoco|instancevg|Qwen3.8,learned,0.99,0.9658282052009188,0.7472421518305306,31452.77611075156,3995.4075477114216 +grefcoco|instancevg|Qwen3.8,learned,1.0,1.0,0.7447182599325072,32565.206836791,4134.535350192891 +grefcoco|instancevg|Qwen3.8,confidence,0.0,0.0,0.6695119530385957,7.502680064,46.08042127620138 +grefcoco|instancevg|Qwen3.8,confidence,0.01,0.002467175226157729,0.6686895612965431,87.81925530502161,56.12533315660624 +grefcoco|instancevg|Qwen3.8,confidence,0.02,0.004849275444516916,0.6680373195700876,165.3662934687666,65.823868765273 +grefcoco|instancevg|Qwen3.8,confidence,0.03,0.007600034030003119,0.6675835861951621,254.9146589673769,77.02336821813819 +grefcoco|instancevg|Qwen3.8,confidence,0.04,0.009811984232765221,0.6671014944843037,326.9226229765687,86.02915128332876 +grefcoco|instancevg|Qwen3.8,confidence,0.05,0.012534384482318578,0.6666194027734452,415.54780944942013,97.11319197894791 +grefcoco|instancevg|Qwen3.8,confidence,0.06,0.01607917647392451,0.6662223860703854,530.9451876692788,111.54553663470202 +grefcoco|instancevg|Qwen3.8,confidence,0.07,0.019709043473328986,0.6658253693673255,649.112102966414,126.32425756219423 +grefcoco|instancevg|Qwen3.8,confidence,0.08,0.02353741882426339,0.6655985026798628,773.7412714438613,141.91118979040866 +grefcoco|instancevg|Qwen3.8,confidence,0.09,0.028868785979638715,0.6651164109690043,947.2989282865286,163.61743615266283 +grefcoco|instancevg|Qwen3.8,confidence,0.1,0.0345971698380739,0.6647761109378102,1133.7810914898203,186.94010511636148 +grefcoco|instancevg|Qwen3.8,confidence,0.11,0.03981510364971783,0.664435810906616,1303.6460322294522,208.18451644963153 +grefcoco|instancevg|Qwen3.8,confidence,0.12,0.045855429203414344,0.664067152539489,1500.2831647160913,232.77723174303654 +grefcoco|instancevg|Qwen3.8,confidence,0.13,0.05317187987408899,0.6637552108442276,1738.4633533618796,262.565591112513 +grefcoco|instancevg|Qwen3.8,confidence,0.14,0.05960922213084536,0.6636134191645634,1948.0249922091427,288.77472900736245 +grefcoco|instancevg|Qwen3.8,confidence,0.15,0.0675211978561098,0.6632164024615036,2205.591940395867,320.98772227900565 +grefcoco|instancevg|Qwen3.8,confidence,0.16,0.07551824858917279,0.6626208774069138,2465.928425659868,353.54709182238696 +grefcoco|instancevg|Qwen3.8,confidence,0.17,0.08212574086152624,0.6622522190397867,2681.0291386616846,380.44898226071257 +grefcoco|instancevg|Qwen3.8,confidence,0.18,0.08856308311828262,0.6619969940163911,2890.590777508948,406.6581201555621 +grefcoco|instancevg|Qwen3.8,confidence,0.19,0.09480191702350906,0.6615149023055327,3093.6901631758988,432.0590467496893 +grefcoco|instancevg|Qwen3.8,confidence,0.2,0.10007656750701868,0.6612596772821371,3265.4014619670484,453.5343755974514 +grefcoco|instancevg|Qwen3.8,confidence,0.21,0.10554972634205825,0.6608343022431443,3443.575013938511,475.81791574593575 +grefcoco|instancevg|Qwen3.8,confidence,0.22,0.11005870175538099,0.6605507188838159,3590.3604790341706,494.17585814805494 +grefcoco|instancevg|Qwen3.8,confidence,0.23,0.11527663556702493,0.6600969855088903,3760.2254197738025,515.420269481325 +grefcoco|instancevg|Qwen3.8,confidence,0.24,0.12029606102713893,0.6598417604854947,3923.6281073331224,535.8564695138729 +grefcoco|instancevg|Qwen3.8,confidence,0.25,0.12477667810452883,0.6594731021183677,4069.490393403024,554.0989531587461 +grefcoco|instancevg|Qwen3.8,confidence,0.26,0.12914386183818732,0.6590193687434421,4211.659963369889,571.879601774635 +grefcoco|instancevg|Qwen3.8,confidence,0.27,0.1322916371267334,0.658679068712248,4314.132835229124,584.6955238289447 +grefcoco|instancevg|Qwen3.8,confidence,0.28,0.13626180415733205,0.6582820520091881,4443.377898835366,600.8597498433894 +grefcoco|instancevg|Qwen3.8,confidence,0.29,0.14023197118793068,0.657743243626464,4572.622962441607,617.0239758578339 +grefcoco|instancevg|Qwen3.8,confidence,0.3,0.14326631313274538,0.657374585259337,4671.403118197806,629.3780628831595 +grefcoco|instancevg|Qwen3.8,confidence,0.31,0.14766185520233674,0.6569775685562771,4814.495867190431,647.2741702562946 +grefcoco|instancevg|Qwen3.8,confidence,0.32,0.15066783881121856,0.6567507018688143,4912.352843920871,659.512798524374 +grefcoco|instancevg|Qwen3.8,confidence,0.33,0.15452457249808582,0.6565238351813516,5037.905191424078,675.2151895098345 +grefcoco|instancevg|Qwen3.8,confidence,0.34,0.15806936448969175,0.6564104018376201,5153.302569643936,689.6475341655887 +grefcoco|instancevg|Qwen3.8,confidence,0.35,0.16201117318435754,0.6562686101579559,5281.6244542244185,705.6963014227872 +grefcoco|instancevg|Qwen3.8,confidence,0.36,0.1657544735274934,0.6558999517908289,5403.484085624589,720.9368573792635 +grefcoco|instancevg|Qwen3.8,confidence,0.37,0.17012165726115192,0.6558715934548961,5545.653655591455,738.7175059951526 +grefcoco|instancevg|Qwen3.8,confidence,0.38,0.17536794940872868,0.6555596517596347,5716.441775356846,760.0773760856687 +grefcoco|instancevg|Qwen3.8,confidence,0.39,0.17965005813458867,0.6553044267362391,5855.841808246436,777.5116484298197 +grefcoco|instancevg|Qwen3.8,confidence,0.4,0.18401724186824717,0.6552193517284406,5998.011378213301,795.2922970457087 +grefcoco|instancevg|Qwen3.8,confidence,0.41,0.18858293395343562,0.6552760684003063,6146.643201360479,813.88115696232 +grefcoco|instancevg|Qwen3.8,confidence,0.42,0.19238295096843716,0.6554745767518362,6270.349190812167,829.3526304332884 +grefcoco|instancevg|Qwen3.8,confidence,0.43,0.19689192638175992,0.655502935087769,6417.134655907828,847.7105728354077 +grefcoco|instancevg|Qwen3.8,confidence,0.44,0.20182627683407536,0.6555880100955676,6577.76780638987,867.8003965962173 +grefcoco|instancevg|Qwen3.8,confidence,0.45,0.20718600232538353,0.6559283101267618,6752.248642258297,889.6221017157176 +grefcoco|instancevg|Qwen3.8,confidence,0.46,0.21189348609023623,0.656070101806426,6905.49636053427,908.7882554185591 +grefcoco|instancevg|Qwen3.8,confidence,0.47,0.217139778237813,0.6567790602047472,7076.28448029966,930.148125509075 +grefcoco|instancevg|Qwen3.8,confidence,0.48,0.22198905368232993,0.6584238436888523,7234.148093704427,949.8915729981468 +grefcoco|instancevg|Qwen3.8,confidence,0.49,0.22763236253296656,0.6604372855400845,7417.860719830442,972.8678656901072 +grefcoco|instancevg|Qwen3.8,confidence,0.5,0.2345234381646485,0.662762669086578,7642.1932230898465,1000.9243437008932 +grefcoco|instancevg|Qwen3.8,confidence,0.51,0.24382497235062248,0.6654567110001985,7944.995943538756,1038.7948160775923 +grefcoco|instancevg|Qwen3.8,confidence,0.52,0.252644414825738,0.6678955278904234,8232.104620549764,1074.7024895811082 +grefcoco|instancevg|Qwen3.8,confidence,0.53,0.26401610753480986,0.6703627031165811,8602.299409879071,1121.0014512367675 +grefcoco|instancevg|Qwen3.8,confidence,0.54,0.2760116836344043,0.6731134617020673,8992.804137775072,1169.8405055518392 +grefcoco|instancevg|Qwen3.8,confidence,0.55,0.2893401015228426,0.6758358619516207,9426.69827988174,1224.1061214574747 +grefcoco|instancevg|Qwen3.8,confidence,0.56,0.3036043444970649,0.679295578935428,9891.057329838453,1282.1818763522294 +grefcoco|instancevg|Qwen3.8,confidence,0.57,0.3175850041119587,0.6822448458724442,10346.184589537575,1339.1030436745236 +grefcoco|instancevg|Qwen3.8,confidence,0.58,0.33462836400760004,0.6853926211609903,10901.015184018655,1408.4937567793893 +grefcoco|instancevg|Qwen3.8,confidence,0.59,0.35637920766809406,0.6889090548166633,11609.09349677571,1497.0506235870967 +grefcoco|instancevg|Qwen3.8,confidence,0.6,0.37968975980489467,0.6922553384567394,12367.9466559495,1591.9577220433357 +grefcoco|instancevg|Qwen3.8,confidence,0.61,0.4046450954258004,0.6949493803703599,13180.344198617304,1693.5614284198446 +grefcoco|instancevg|Qwen3.8,confidence,0.62,0.43141536454640844,0.6983807390182344,14051.825198933677,1802.5544952600997 +grefcoco|instancevg|Qwen3.8,confidence,0.63,0.4610214672603012,0.7015568726427134,15015.624101825935,1923.093437824958 +grefcoco|instancevg|Qwen3.8,confidence,0.64,0.4916484700677764,0.7053285313217821,16012.657449645512,2047.7888956506733 +grefcoco|instancevg|Qwen3.8,confidence,0.65,0.5209426310864078,0.7086180982899923,16966.301383254424,2167.0577918858253 +grefcoco|instancevg|Qwen3.8,confidence,0.66,0.5529308340186598,0.71148229021921,18007.64732431043,2297.2952700593505 +grefcoco|instancevg|Qwen3.8,confidence,0.67,0.5888041289737118,0.7155091739216742,19175.4687918954,2443.350597975582 +grefcoco|instancevg|Qwen3.8,confidence,0.68,0.6175027649377535,0.7182882908430933,20109.725965963375,2560.1948603085675 +grefcoco|instancevg|Qwen3.8,confidence,0.69,0.6421461588633979,0.7211241244363781,20911.968539347832,2660.52852035537 +grefcoco|instancevg|Qwen3.8,confidence,0.7,0.665910444375124,0.7233077163032073,21685.59256293376,2757.282958927545 +grefcoco|instancevg|Qwen3.8,confidence,0.71,0.6838612710206166,0.7243286163967898,22269.96488623913,2830.3683522642846 +grefcoco|instancevg|Qwen3.8,confidence,0.72,0.7029464311034228,0.7260017582168279,22891.264370574845,2908.072095890864 +grefcoco|instancevg|Qwen3.8,confidence,0.73,0.7180614241556306,0.7278450500524629,23383.318791304326,2969.611613503 +grefcoco|instancevg|Qwen3.8,confidence,0.74,0.7352465757309361,0.7297450585599637,23942.7652809142,3039.579620394096 +grefcoco|instancevg|Qwen3.8,confidence,0.75,0.7478093185491875,0.7311629753566061,24351.733589325377,3090.727849854088 +grefcoco|instancevg|Qwen3.8,confidence,0.76,0.759804894648782,0.7328928338485098,24742.238317221378,3139.5669041691604 +grefcoco|instancevg|Qwen3.8,confidence,0.77,0.7729064458497575,0.7342540339732865,25168.74702712198,3192.9088500168273 +grefcoco|instancevg|Qwen3.8,confidence,0.78,0.7855259053398747,0.735274934066869,25579.561693584674,3244.287996991312 +grefcoco|instancevg|Qwen3.8,confidence,0.79,0.7964722230099538,0.7362958341604514,25935.908797527598,3288.855077288281 +grefcoco|instancevg|Qwen3.8,confidence,0.8,0.8082125740861527,0.7377704676289595,26318.104914191772,3336.6550027881385 +grefcoco|instancevg|Qwen3.8,confidence,0.81,0.8197544168108215,0.738422709355415,26693.838777675628,3383.6467169872735 +grefcoco|instancevg|Qwen3.8,confidence,0.82,0.8333380597226555,0.7389048010662734,27136.04153101413,3438.9514617081236 +grefcoco|instancevg|Qwen3.8,confidence,0.83,0.8426112355726966,0.7398689844879902,27437.921072437275,3476.7064753275763 +grefcoco|instancevg|Qwen3.8,confidence,0.84,0.8529903865241187,0.7403227178629158,27775.804595865022,3518.964380479624 +grefcoco|instancevg|Qwen3.8,confidence,0.85,0.8635396874911381,0.7406913762300428,28119.227193447325,3561.9150381751488 +grefcoco|instancevg|Qwen3.8,confidence,0.86,0.8710262881774098,0.7413152596205654,28362.946456247664,3592.3961500881014 +grefcoco|instancevg|Qwen3.8,confidence,0.87,0.8809517057539064,0.7422227263704165,28686.05911526327,3632.8067151242126 +grefcoco|instancevg|Qwen3.8,confidence,0.88,0.8913024983693957,0.7430167597765364,29023.019459665255,3674.9491615190145 +grefcoco|instancevg|Qwen3.8,confidence,0.89,0.900915974250631,0.743442134815529,29335.977149397513,3714.0896802254197 +grefcoco|instancevg|Qwen3.8,confidence,0.9,0.910954825170859,0.7443212432294473,29662.782524516155,3754.9620802905156 +grefcoco|instancevg|Qwen3.8,confidence,0.91,0.921901142840938,0.7448033349403057,30019.129628459075,3799.529160587484 +grefcoco|instancevg|Qwen3.8,confidence,0.92,0.9303235686129938,0.7450018432918356,30293.31379910946,3833.820411489556 +grefcoco|instancevg|Qwen3.8,confidence,0.93,0.9393698777755721,0.7448884099481042,30587.80790832654,3870.6517550510403 +grefcoco|instancevg|Qwen3.8,confidence,0.94,0.9490117119927403,0.7448884099481042,30901.688777084553,3909.907732514692 +grefcoco|instancevg|Qwen3.8,confidence,0.95,0.9587953378895726,0.7449734849559028,31220.185540971364,3949.7410037645727 +grefcoco|instancevg|Qwen3.8,confidence,0.96,0.9673595553412926,0.7452570683152313,31498.985606750543,3984.609548452875 +grefcoco|instancevg|Qwen3.8,confidence,0.97,0.9746192893401016,0.745313784987097,31735.319437344813,4014.1669903078596 +grefcoco|instancevg|Qwen3.8,confidence,0.98,0.9830984317840229,0.7450585599637013,32011.349966046713,4048.689158724423 +grefcoco|instancevg|Qwen3.8,confidence,0.99,0.9908118991577575,0.7448884099481042,32262.45466105313,4080.0939406953444 +grefcoco|instancevg|Qwen3.8,confidence,1.0,1.0,0.7447182599325072,32561.564665399,4117.502578043059 +construction|yolo26x|Qwen3.8,learned,0.0,0.0,0.18129624718318377,139.16228659200002,71.56936161300109 +construction|yolo26x|Qwen3.8,learned,0.01,0.013648468708388815,0.18693471712069915,1461.3537694872334,205.6311162003908 +construction|yolo26x|Qwen3.8,learned,0.02,0.019640479360852198,0.18781313491575913,2041.8280790509946,264.48749626314725 +construction|yolo26x|Qwen3.8,learned,0.03,0.031291611185086554,0.1913128749335621,3170.5281254249744,378.9304574962848 +construction|yolo26x|Qwen3.8,learned,0.04,0.04061251664447404,0.19266008228682668,4073.4881625241583,470.48482648279486 +construction|yolo26x|Qwen3.8,learned,0.05,0.045938748335552594,0.19531128485766117,4589.465326580834,522.8016087608005 +construction|yolo26x|Qwen3.8,learned,0.06,0.0559254327563249,0.19800731707634547,5556.922509187103,620.8955755320613 +construction|yolo26x|Qwen3.8,learned,0.07,0.07456724367509987,0.19763843353437874,7362.842583385471,804.0043135050814 +construction|yolo26x|Qwen3.8,learned,0.08,0.08688415446071904,0.1974112776278713,8556.039775266534,924.9868725229696 +construction|yolo26x|Qwen3.8,learned,0.09,0.09587217043941411,0.20030220967092682,9426.751239612175,1013.2714426171043 +construction|yolo26x|Qwen3.8,learned,0.1,0.09920106524633822,0.20211074868345033,9749.236967147599,1045.9694315408578 +construction|yolo26x|Qwen3.8,learned,0.11,0.11284953395472704,0.20834032699131655,11071.428450042833,1180.0311861282478 +construction|yolo26x|Qwen3.8,learned,0.12,0.12350199733688415,0.21121674052765757,12103.382778156185,1284.664750684259 +construction|yolo26x|Qwen3.8,learned,0.13,0.13315579227696406,0.21276215593410724,13038.591388008912,1379.4889185631446 +construction|yolo26x|Qwen3.8,learned,0.14,0.14347536617842876,0.21511521078428622,14038.297143368722,1480.8526842267806 +construction|yolo26x|Qwen3.8,learned,0.15,0.15246338215712382,0.21738179676765032,14909.008607714362,1569.1372543209152 +construction|yolo26x|Qwen3.8,learned,0.16,0.1604527296937417,0.22260454951009195,15682.974353799378,1647.612427737924 +construction|yolo26x|Qwen3.8,learned,0.17,0.1780958721704394,0.22627631882842034,17392.148709737117,1820.9117690338178 +construction|yolo26x|Qwen3.8,learned,0.18,0.18442077230359522,0.2270967395967396,18004.871592054424,1883.0379479889498 +construction|yolo26x|Qwen3.8,learned,0.19,0.2000665778961385,0.23397114982462924,19520.55451147091,2036.7184959305916 +construction|yolo26x|Qwen3.8,learned,0.2,0.20439414114513982,0.23512276426997616,19939.78595726696,2079.2258815314713 +construction|yolo26x|Qwen3.8,learned,0.21,0.21937416777629826,0.24202469545559713,21390.97173117636,2226.3668316883623 +construction|yolo26x|Qwen3.8,learned,0.22,0.22769640479360853,0.24353835097564847,22197.18605001492,2308.1118039977464 +construction|yolo26x|Qwen3.8,learned,0.23,0.23601864181091878,0.2454657390209296,23003.400368853476,2389.8567763071305 +construction|yolo26x|Qwen3.8,learned,0.24,0.24533954727030627,0.2483414219175777,23906.36040595266,2481.4111452936404 +construction|yolo26x|Qwen3.8,learned,0.25,0.25432756324900135,0.2504846177578749,24777.071870298303,2569.695715387775 +construction|yolo26x|Qwen3.8,learned,0.26,0.2636484687083888,0.252142216376585,25680.031907397482,2661.250084374285 +construction|yolo26x|Qwen3.8,learned,0.27,0.275965379494008,0.25276746625574176,26873.22909927855,2782.2326433921735 +construction|yolo26x|Qwen3.8,learned,0.28,0.29427430093209056,0.25702642690899347,28646.900600723373,2962.071582472818 +construction|yolo26x|Qwen3.8,learned,0.29,0.31091877496671105,0.26183035419940204,30259.32923840049,3125.561527091586 +construction|yolo26x|Qwen3.8,learned,0.3,0.322237017310253,0.2672887365132105,31355.780712020925,3236.734689432348 +construction|yolo26x|Qwen3.8,learned,0.31,0.33388814913448733,0.27039536303602457,32484.480758394904,3351.1776506654855 +construction|yolo26x|Qwen3.8,learned,0.32,0.34354194407456723,0.2713201570387918,33419.68936824763,3446.001818544371 +construction|yolo26x|Qwen3.8,learned,0.33,0.3492010652463382,0.2722481828097158,33967.91510505785,3501.588399714752 +construction|yolo26x|Qwen3.8,learned,0.34,0.3545272969374168,0.274417471554269,34483.89226911453,3553.905181992758 +construction|yolo26x|Qwen3.8,learned,0.35,0.3648468708388815,0.2782223498666452,35483.59802447434,3655.268947656394 +construction|yolo26x|Qwen3.8,learned,0.36,0.37217043941411454,0.2795152621397537,36193.06662505227,3727.204523288652 +construction|yolo26x|Qwen3.8,learned,0.37,0.37683089214380827,0.28090947933574756,36644.54664360186,3772.981707781907 +construction|yolo26x|Qwen3.8,learned,0.38,0.3854860186418109,0.28295009794869297,37483.00953519396,3857.9964789836663 +construction|yolo26x|Qwen3.8,learned,0.39,0.39980026631158455,0.28644294895435674,38869.69816359627,3998.5978313558066 +construction|yolo26x|Qwen3.8,learned,0.4,0.4054593874833555,0.2862200047733393,39417.923900406495,4054.1844125261878 +construction|yolo26x|Qwen3.8,learned,0.41,0.4104527296937417,0.28760386086589435,39901.65249170963,4103.231395911818 +construction|yolo26x|Qwen3.8,learned,0.42,0.4181091877496671,0.2930744270093899,40643.3696650411,4178.436770436451 +construction|yolo26x|Qwen3.8,learned,0.43,0.42809587217043943,0.2961685540536353,41610.82684764737,4276.530737207712 +construction|yolo26x|Qwen3.8,learned,0.44,0.4370838881491345,0.2977058723546182,42481.53831199301,4364.8153073018475 +construction|yolo26x|Qwen3.8,learned,0.45,0.44673768308921435,0.2999402430635043,43416.74692184573,4459.639475180732 +construction|yolo26x|Qwen3.8,learned,0.46,0.45206391478029295,0.2993046120720782,43932.72408590242,4511.956257458738 +construction|yolo26x|Qwen3.8,learned,0.47,0.45872170439414117,0.2998310238962732,44577.69554097326,4577.352235306245 +construction|yolo26x|Qwen3.8,learned,0.48,0.4673768308921438,0.30393211265289466,45416.15843256536,4662.367006508005 +construction|yolo26x|Qwen3.8,learned,0.49,0.47503328894806923,0.30350001249166053,46157.87560589683,4737.572381032637 +construction|yolo26x|Qwen3.8,learned,0.5,0.4876830892143808,0.29875800068449215,47383.321370531434,4861.824738942901 +construction|yolo26x|Qwen3.8,learned,0.51,0.5046604527296937,0.29989697064322696,49027.99858096209,5028.5844824540445 +construction|yolo26x|Qwen3.8,learned,0.52,0.5206391478029294,0.297424793956044,50575.93007313212,5185.534829288062 +construction|yolo26x|Qwen3.8,learned,0.53,0.5249667110519307,0.297424793956044,50995.161518928166,5228.042214888941 +construction|yolo26x|Qwen3.8,learned,0.54,0.5436085219707057,0.2978194775414123,52801.08159312654,5411.150952861962 +construction|yolo26x|Qwen3.8,learned,0.55,0.5529294274300932,0.29630990060928936,53704.04163022572,5502.705321848472 +construction|yolo26x|Qwen3.8,learned,0.56,0.5612516644474035,0.29578344205458756,54510.25594906428,5584.450294157856 +construction|yolo26x|Qwen3.8,learned,0.57,0.5692410119840213,0.2950949302773625,55284.221695149296,5662.925467574864 +construction|yolo26x|Qwen3.8,learned,0.58,0.5902130492676432,0.2947373376338526,57315.88177862246,5868.922797794512 +construction|yolo26x|Qwen3.8,learned,0.59,0.6021970705725699,0.29722091277674884,58476.830397749975,5986.635557920024 +construction|yolo26x|Qwen3.8,learned,0.6,0.6208388814913449,0.29665021956289706,60282.750471948355,6169.744295893045 +construction|yolo26x|Qwen3.8,learned,0.61,0.6288282290279628,0.295988506182091,61056.71621803337,6248.219469310054 +construction|yolo26x|Qwen3.8,learned,0.62,0.6421438082556591,0.2939669175161438,62346.65912817505,6379.011425005067 +construction|yolo26x|Qwen3.8,learned,0.63,0.6491344873501997,0.29277362668991463,63023.87915599944,6447.67720174495 +construction|yolo26x|Qwen3.8,learned,0.64,0.6571238348868176,0.2930821542583563,63797.84490208446,6526.152375161959 +construction|yolo26x|Qwen3.8,learned,0.65,0.6677762982689747,0.2892305220657417,64829.79923019781,6630.78593971797 +construction|yolo26x|Qwen3.8,learned,0.66,0.6744340878828229,0.2915915088317972,65474.77068526865,6696.181917565477 +construction|yolo26x|Qwen3.8,learned,0.67,0.6854194407456724,0.2936540392680743,66538.97358613556,6804.085281013864 +construction|yolo26x|Qwen3.8,learned,0.68,0.7047270306258322,0.29632332439424064,68409.390805841,6993.733616771635 +construction|yolo26x|Qwen3.8,learned,0.69,0.7157123834886817,0.2937246709263134,69473.5937067079,7101.6369802200215 +construction|yolo26x|Qwen3.8,learned,0.7,0.7253661784287616,0.2952254059747369,70408.80231656063,7196.461148098907 +construction|yolo26x|Qwen3.8,learned,0.71,0.7283621837549934,0.2945683782087344,70699.03947134252,7225.889338130286 +construction|yolo26x|Qwen3.8,learned,0.72,0.741011984021305,0.29252963134197957,71924.48523597712,7350.1416960405495 +construction|yolo26x|Qwen3.8,learned,0.73,0.7526631158455392,0.2900395106945919,73053.1852823511,7464.584657273686 +construction|yolo26x|Qwen3.8,learned,0.74,0.7616511318242344,0.2887744568626224,73923.89674669674,7552.869227367822 +construction|yolo26x|Qwen3.8,learned,0.75,0.7709720372836218,0.28730364578560735,74826.85678379593,7644.4235963543315 +construction|yolo26x|Qwen3.8,learned,0.76,0.7749667110519307,0.2858070539975931,75213.83965683842,7683.661183062835 +construction|yolo26x|Qwen3.8,learned,0.77,0.7792942743009321,0.28488735368132817,75633.07110263448,7726.168568663716 +construction|yolo26x|Qwen3.8,learned,0.78,0.7846205059920106,0.2830658409372236,76149.04826669116,7778.485350941721 +construction|yolo26x|Qwen3.8,learned,0.79,0.7972703062583223,0.2809995077380739,77374.49403132577,7902.737708851985 +construction|yolo26x|Qwen3.8,learned,0.8,0.8129161118508655,0.28013559922601305,78890.17695074226,8056.418256793627 +construction|yolo26x|Qwen3.8,learned,0.81,0.8199067909454061,0.27814841049220523,79567.39697856663,8125.084033533509 +construction|yolo26x|Qwen3.8,learned,0.82,0.831225033288948,0.2772348763322643,80663.84845218707,8236.25719587427 +construction|yolo26x|Qwen3.8,learned,0.83,0.84254327563249,0.27700595116508253,81760.29992580751,8347.430358215033 +construction|yolo26x|Qwen3.8,learned,0.84,0.857190412782956,0.2763733409861609,83179.23712696337,8491.301509479548 +construction|yolo26x|Qwen3.8,learned,0.85,0.8618508655126498,0.2761345157638265,83630.71714551296,8537.078693972804 +construction|yolo26x|Qwen3.8,learned,0.86,0.87017310252996,0.2761345157638265,84436.93146435152,8618.823666282187 +construction|yolo26x|Qwen3.8,learned,0.87,0.8848202396804261,0.27546832609774896,85855.86866550738,8762.694817546704 +construction|yolo26x|Qwen3.8,learned,0.88,0.8984687083888149,0.27423981143437465,87178.06014840261,8896.756572134092 +construction|yolo26x|Qwen3.8,learned,0.89,0.9087882822902796,0.2736575311942959,88177.76590376243,8998.12033779773 +construction|yolo26x|Qwen3.8,learned,0.9,0.9161118508655126,0.272975172975173,88887.23450434035,9070.055913429987 +construction|yolo26x|Qwen3.8,learned,0.91,0.9217709720372836,0.2727190231806067,89435.46024115058,9125.64249460037 +construction|yolo26x|Qwen3.8,learned,0.92,0.9264314247669774,0.27264729817760425,89886.94025970016,9171.419679093624 +construction|yolo26x|Qwen3.8,learned,0.93,0.933422103861518,0.2741441928343702,90564.16028752456,9240.085455833507 +construction|yolo26x|Qwen3.8,learned,0.94,0.9417443408788282,0.2733716828250215,91370.37460636311,9321.83042814289 +construction|yolo26x|Qwen3.8,learned,0.95,0.9530625832223701,0.2727312493845365,92466.82607998354,9433.003590483651 +construction|yolo26x|Qwen3.8,learned,0.96,0.9613848202396804,0.27223682114986464,93273.04039882211,9514.748562793036 +construction|yolo26x|Qwen3.8,learned,0.97,0.9673768308921438,0.27453093162048386,93853.51470838586,9573.604942855793 +construction|yolo26x|Qwen3.8,learned,0.98,0.9760319573901465,0.2737331615154065,94691.97759997797,9658.619714057553 +construction|yolo26x|Qwen3.8,learned,0.99,0.9890146471371505,0.2725383412883413,95949.67193736612,9786.141870860192 +construction|yolo26x|Qwen3.8,learned,1.0,1.0,0.27001009067886245,97013.87483823301,9894.045234308578 +construction|yolo26x|Qwen3.8,confidence,0.0,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.01,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.02,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.03,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.04,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.05,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.06,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.07,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.08,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.09,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.1,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.11,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.12,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.13,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.14,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.15,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.16,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.17,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.18,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.19,0.17942743009320905,0.1784481207469488,17517.959245999285,1808.7296972439362 +construction|yolo26x|Qwen3.8,confidence,0.2,0.19340878828229027,0.1780928651788194,18872.39930164806,1946.0612507237013 +construction|yolo26x|Qwen3.8,confidence,0.21,0.20306258322237017,0.17829989216687975,19807.60791150079,2040.8854186025867 +construction|yolo26x|Qwen3.8,confidence,0.22,0.20838881491344874,0.1779854683890517,20323.585075557465,2093.2022008805925 +construction|yolo26x|Qwen3.8,confidence,0.23,0.21537949400798934,0.17917657728013692,21000.805103381852,2161.867977620475 +construction|yolo26x|Qwen3.8,confidence,0.24,0.23135818908122505,0.17803687432776102,22548.736595551884,2318.8183244544925 +construction|yolo26x|Qwen3.8,confidence,0.25,0.24800266311584554,0.18026802885144627,24161.165233228996,2482.3082690732604 +construction|yolo26x|Qwen3.8,confidence,0.26,0.2589880159786951,0.18119827871024874,25225.368134095894,2590.211632521647 +construction|yolo26x|Qwen3.8,confidence,0.27,0.27430093209054596,0.17963665085316138,26708.802480758837,2740.6223815709136 +construction|yolo26x|Qwen3.8,confidence,0.28,0.281624500665779,0.17924095839307458,27418.271081336767,2812.5579572031716 +construction|yolo26x|Qwen3.8,confidence,0.29,0.28761651131824234,0.1789157174618718,27998.745390900527,2871.4143372659278 +construction|yolo26x|Qwen3.8,confidence,0.3,0.2969374167776298,0.1787206845800119,28901.70542799971,2962.9687062524376 +construction|yolo26x|Qwen3.8,confidence,0.31,0.30459387483355527,0.17813242189706796,29643.422601331185,3038.174080777071 +construction|yolo26x|Qwen3.8,confidence,0.32,0.3129161118508655,0.17772000842137498,30449.636920169738,3119.919053086455 +construction|yolo26x|Qwen3.8,confidence,0.33,0.31990679094540614,0.17791259734473341,31126.85694799413,3188.5848298263377 +construction|yolo26x|Qwen3.8,confidence,0.34,0.3322237017310253,0.1789653934599337,32320.05413987519,3309.5673888442257 +construction|yolo26x|Qwen3.8,confidence,0.35,0.34054593874833555,0.18042682200794885,33126.268458713756,3391.31236115361 +construction|yolo26x|Qwen3.8,confidence,0.36,0.3488681757656458,0.17962877325912951,33932.48277755231,3473.057333462994 +construction|yolo26x|Qwen3.8,confidence,0.37,0.3581890812250333,0.18158304685510065,34835.44281465149,3564.611702449504 +construction|yolo26x|Qwen3.8,confidence,0.38,0.37017310252996005,0.18069239723097547,35996.39143377901,3682.3244625750167 +construction|yolo26x|Qwen3.8,confidence,0.39,0.37683089214380827,0.1812845803851709,36641.362888849864,3747.720440422524 +construction|yolo26x|Qwen3.8,confidence,0.4,0.38348868175765644,0.1824781664689562,37286.3343439207,3813.116418270031 +construction|yolo26x|Qwen3.8,confidence,0.41,0.4021304926764314,0.18220594501399923,39092.254418119075,3996.225156243051 +construction|yolo26x|Qwen3.8,confidence,0.42,0.41311584553928093,0.18162498930772586,40156.457318985966,4104.128519691438 +construction|yolo26x|Qwen3.8,confidence,0.43,0.42676431424766975,0.18066434967152634,41478.6488018812,4238.190274278827 +construction|yolo26x|Qwen3.8,confidence,0.44,0.4324234354194407,0.1803913789289444,42026.874538691416,4293.776855449209 +construction|yolo26x|Qwen3.8,confidence,0.45,0.44107856191744343,0.17959468902865128,42865.33743028352,4378.791626650968 +construction|yolo26x|Qwen3.8,confidence,0.46,0.45772303595206393,0.17863383514615866,44477.766067960634,4542.281571269737 +construction|yolo26x|Qwen3.8,confidence,0.47,0.4717043941411451,0.17957308538379646,45832.20612360941,4679.6131247495005 +construction|yolo26x|Qwen3.8,confidence,0.48,0.4826897470039947,0.17910245145856155,46896.40902447631,4787.516488197888 +construction|yolo26x|Qwen3.8,confidence,0.49,0.4896804260985353,0.17871833907402393,47573.62905230069,4856.182264937771 +construction|yolo26x|Qwen3.8,confidence,0.5,0.5013315579227696,0.18015785955296285,48702.32909867467,4970.625226170908 +construction|yolo26x|Qwen3.8,confidence,0.51,0.5116511318242344,0.1810878167493375,49702.034854034486,5071.988991834544 +construction|yolo26x|Qwen3.8,confidence,0.52,0.5183089214380826,0.18177962418520882,50347.00630910533,5137.384969682052 +construction|yolo26x|Qwen3.8,confidence,0.53,0.5252996005326231,0.18246323539519374,51024.226336929714,5206.050746421934 +construction|yolo26x|Qwen3.8,confidence,0.54,0.5396138482023968,0.1839876166906015,52410.914965332035,5346.652098794074 +construction|yolo26x|Qwen3.8,confidence,0.55,0.5452729693741678,0.1846674213907964,52959.14070214226,5402.2386799644555 +construction|yolo26x|Qwen3.8,confidence,0.56,0.559254327563249,0.18676294233791663,54313.58075779103,5539.57023344422 +construction|yolo26x|Qwen3.8,confidence,0.57,0.572237017310253,0.18938663221175195,55571.275095179175,5667.092390246859 +construction|yolo26x|Qwen3.8,confidence,0.58,0.5862183754993342,0.1909571002676625,56925.71515082795,5804.423943726624 +construction|yolo26x|Qwen3.8,confidence,0.59,0.5998668442077231,0.1934300101790686,58247.90663372319,5938.485698314014 +construction|yolo26x|Qwen3.8,confidence,0.6,0.6191744340878829,0.1973701682130997,60118.32385342864,6128.134034071785 +construction|yolo26x|Qwen3.8,confidence,0.61,0.6264980026631158,0.1986727924998457,60827.792454006565,6200.069609704043 +construction|yolo26x|Qwen3.8,confidence,0.62,0.6384820239680427,0.20186289191698192,61988.74107313409,6317.782369829556 +construction|yolo26x|Qwen3.8,confidence,0.63,0.6461384820239681,0.20246031058520178,62730.45824646557,6392.987744354189 +construction|yolo26x|Qwen3.8,confidence,0.64,0.653462050599201,0.20349944137622245,63439.92684704349,6464.923319986447 +construction|yolo26x|Qwen3.8,confidence,0.65,0.6597869507323568,0.20442990090722246,64052.64972936079,6527.049498941578 +construction|yolo26x|Qwen3.8,confidence,0.66,0.6661118508655126,0.2044993942500819,64665.372611678096,6589.17567789671 +construction|yolo26x|Qwen3.8,confidence,0.67,0.6734354194407457,0.2064379542733177,65374.84121225603,6661.1112535289685 +construction|yolo26x|Qwen3.8,confidence,0.68,0.6820905459387483,0.2065831206227621,66213.30410384812,6746.126024730727 +construction|yolo26x|Qwen3.8,confidence,0.69,0.6927430093209055,0.20802802394105854,67245.25843196148,6850.759589286739 +construction|yolo26x|Qwen3.8,confidence,0.7,0.7090545938748336,0.21169680596278434,68825.43849688505,7010.979735013132 +construction|yolo26x|Qwen3.8,confidence,0.71,0.7170439414114514,0.2139280527103987,69599.40424297006,7089.454908430141 +construction|yolo26x|Qwen3.8,confidence,0.72,0.7243675099866844,0.2146900231025185,70308.87284354799,7161.390484062398 +construction|yolo26x|Qwen3.8,confidence,0.73,0.7330226364846871,0.21653561577692484,71147.33573514009,7246.405255264158 +construction|yolo26x|Qwen3.8,confidence,0.74,0.7363515312916112,0.21721689648133474,71469.82146267552,7279.1032441879115 +construction|yolo26x|Qwen3.8,confidence,0.75,0.7486684420772304,0.22103445955201576,72663.01865455658,7400.0858032057995 +construction|yolo26x|Qwen3.8,confidence,0.76,0.757323568575233,0.22340134248367294,73501.48154614867,7485.100574407558 +construction|yolo26x|Qwen3.8,confidence,0.77,0.7639813581890812,0.2269322568854726,74146.45300121952,7550.496552255066 +construction|yolo26x|Qwen3.8,confidence,0.78,0.7733022636484687,0.22757452798038888,75049.4130383187,7642.050921241575 +construction|yolo26x|Qwen3.8,confidence,0.79,0.784953395472703,0.227821050912831,76178.11308469268,7756.493882474713 +construction|yolo26x|Qwen3.8,confidence,0.8,0.7939414114513982,0.22991865671924006,77048.82454903833,7844.7784525688485 +construction|yolo26x|Qwen3.8,confidence,0.81,0.8039280958721704,0.23486191981046528,78016.28173164459,7942.8724193401085 +construction|yolo26x|Qwen3.8,confidence,0.82,0.8122503328894807,0.23803916047544685,78822.49605048315,8024.617391649493 +construction|yolo26x|Qwen3.8,confidence,0.83,0.8215712383488681,0.24022947987632814,79725.45608758234,8116.171760636003 +construction|yolo26x|Qwen3.8,confidence,0.84,0.8388814913448736,0.24528627405740036,81402.38187076653,8286.201303039523 +construction|yolo26x|Qwen3.8,confidence,0.85,0.8535286284953395,0.249537136326177,82821.3190719224,8430.072454304036 +construction|yolo26x|Qwen3.8,confidence,0.86,0.8638482023968043,0.25211074301800224,83821.0248272822,8531.436219967674 +construction|yolo26x|Qwen3.8,confidence,0.87,0.8745006657789614,0.2540042317315216,84852.97915539556,8636.069784523685 +construction|yolo26x|Qwen3.8,confidence,0.88,0.8764980026631158,0.2540996517958595,85046.4705919168,8655.688577877936 +construction|yolo26x|Qwen3.8,confidence,0.89,0.8841544607190412,0.25436194678023316,85788.18776524828,8730.89395240257 +construction|yolo26x|Qwen3.8,confidence,0.9,0.8891478029294274,0.2534339447803836,86271.91635655142,8779.9409357882 +construction|yolo26x|Qwen3.8,confidence,0.91,0.8981358189081226,0.2563240880787248,87142.62782089706,8868.225505882336 +construction|yolo26x|Qwen3.8,confidence,0.92,0.9054593874833555,0.25551845731687656,87852.09642147498,8940.161081514594 +construction|yolo26x|Qwen3.8,confidence,0.93,0.9221038615179761,0.2597490004762105,89464.5250591521,9103.651026133362 +construction|yolo26x|Qwen3.8,confidence,0.94,0.9254327563249002,0.26306323674307786,89787.01078668753,9136.349015057116 +construction|yolo26x|Qwen3.8,confidence,0.95,0.9364181091877497,0.26218185491449153,90851.21368755442,9244.252378505502 +construction|yolo26x|Qwen3.8,confidence,0.96,0.953728362183755,0.26357052137555514,92528.13947073862,9414.281920909021 +construction|yolo26x|Qwen3.8,confidence,0.97,0.96271637816245,0.26400290307863034,93398.85093508427,9502.566491003154 +construction|yolo26x|Qwen3.8,confidence,0.98,0.9700399467376831,0.2647475724296899,94108.3195356622,9574.502066635414 +construction|yolo26x|Qwen3.8,confidence,0.99,0.987017310252996,0.2649306125590882,95752.99674609285,9741.261810146556 +construction|yolo26x|Qwen3.8,confidence,1.0,1.0,0.27001009067886245,97010.691083481,9868.783966949195 diff --git a/paper_results/data/main_table_cost_qwen_20260912/main_table_results.json b/paper_results/data/main_table_cost_qwen_20260912/main_table_results.json new file mode 100644 index 0000000000000000000000000000000000000000..ef8e3f2eb2ef3bd0d56909cef076cc4eabac62e8 --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/main_table_results.json @@ -0,0 +1,10760 @@ +{ + "source": "C:\\Users\\csh10\\Documents\\CodexProjects\\ICASSP-COLLA\\paper_materials\\draft_20260912_revision19\\data\\results.json", + "protocol": "Confidence and learned routing each select a fixed threshold from the same 1..99% validation budget grid; threshold unchanged on test. No inference or training.", + "groups": [ + { + "key": [ + "grefcoco", + "instancevg", + "Qwen3.8" + ], + "expert_display": "InstanceVG", + "run": "shared_12_instancevg_Qwen3.8", + "validation_rows": 14229, + "test_rows": 35263, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detections_above_07", + "max_score", + "mean_score", + "query_count", + "detections_above_07_valid", + "max_score_valid", + "mean_score_valid", + "query_count_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/instancevg/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 71, + "validation_area_0_25": 0.7477616136060159, + "validation_point": { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.8201560193970061, + "rescued": 1660, + "harmed": 666, + "missed_rescue": 304, + "avoided_harm": 505, + "selected_both_correct": 6505, + "selected_both_wrong": 1271 + }, + "test_point": { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.6838612710206166, + "metric": 0.7243286163967898, + "rescued": 3794, + "harmed": 1861, + "missed_rescue": 1629, + "avoided_harm": 910, + "selected_both_correct": 13678, + "selected_both_wrong": 4782 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7502986857825568, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1964, + "avoided_harm": 1171, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -2.0005727208172175e-07, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.7502284067748963, + "rescued": 0, + "harmed": 1, + "missed_rescue": 1964, + "avoided_harm": 1170, + "selected_both_correct": 141, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -2.4156264544217265e-07, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.7501581277672359, + "rescued": 0, + "harmed": 2, + "missed_rescue": 1964, + "avoided_harm": 1169, + "selected_both_correct": 282, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -2.7729177531909954e-07, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.7498770117365943, + "rescued": 0, + "harmed": 6, + "missed_rescue": 1964, + "avoided_harm": 1165, + "selected_both_correct": 420, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": -3.064858447032748e-07, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.7495958957059526, + "rescued": 0, + "harmed": 10, + "missed_rescue": 1964, + "avoided_harm": 1161, + "selected_both_correct": 559, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": -3.3848826319626824e-07, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.7494553376906318, + "rescued": 0, + "harmed": 12, + "missed_rescue": 1964, + "avoided_harm": 1159, + "selected_both_correct": 699, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": -3.668622525765386e-07, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.7493850586829714, + "rescued": 0, + "harmed": 13, + "missed_rescue": 1964, + "avoided_harm": 1158, + "selected_both_correct": 840, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": -3.98026287484754e-07, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.749314779675311, + "rescued": 0, + "harmed": 14, + "missed_rescue": 1964, + "avoided_harm": 1157, + "selected_both_correct": 982, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": -4.302527258914779e-07, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.7491742216599901, + "rescued": 0, + "harmed": 16, + "missed_rescue": 1964, + "avoided_harm": 1155, + "selected_both_correct": 1122, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": -4.7191912244670675e-07, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.748963384637009, + "rescued": 0, + "harmed": 19, + "missed_rescue": 1964, + "avoided_harm": 1152, + "selected_both_correct": 1261, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": -5.115426802149159e-07, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.7486119895987069, + "rescued": 0, + "harmed": 24, + "missed_rescue": 1964, + "avoided_harm": 1147, + "selected_both_correct": 1398, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": -5.534793672268279e-07, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.7483308735680653, + "rescued": 0, + "harmed": 28, + "missed_rescue": 1964, + "avoided_harm": 1143, + "selected_both_correct": 1537, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": -5.941800509390305e-07, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.7481903155527444, + "rescued": 0, + "harmed": 30, + "missed_rescue": 1964, + "avoided_harm": 1141, + "selected_both_correct": 1677, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": -6.464504735959054e-07, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.7477686415067819, + "rescued": 0, + "harmed": 36, + "missed_rescue": 1964, + "avoided_harm": 1135, + "selected_both_correct": 1813, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": -6.999837864896108e-07, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.7476280834914611, + "rescued": 0, + "harmed": 38, + "missed_rescue": 1964, + "avoided_harm": 1133, + "selected_both_correct": 1954, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": -7.663418841730163e-07, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.7474875254761403, + "rescued": 0, + "harmed": 40, + "missed_rescue": 1964, + "avoided_harm": 1131, + "selected_both_correct": 2094, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": -8.471382670904859e-07, + "operator": "ge", + "actual_fraction": 0.15995502143509735, + "metric": 0.7472766884531591, + "rescued": 0, + "harmed": 43, + "missed_rescue": 1964, + "avoided_harm": 1128, + "selected_both_correct": 2233, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": -9.292756431023008e-07, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.7471361304378382, + "rescued": 0, + "harmed": 45, + "missed_rescue": 1964, + "avoided_harm": 1126, + "selected_both_correct": 2373, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": -1.0245233852401725e-06, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.7470658514301778, + "rescued": 0, + "harmed": 46, + "missed_rescue": 1964, + "avoided_harm": 1125, + "selected_both_correct": 2515, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": -1.1468601996966754e-06, + "operator": "ge", + "actual_fraction": 0.18996415770609318, + "metric": 0.7465036193688945, + "rescued": 0, + "harmed": 54, + "missed_rescue": 1964, + "avoided_harm": 1117, + "selected_both_correct": 2649, + "selected_both_wrong": 0 + }, + { + "budget": 0.2, + "threshold": -1.3018121762797819e-06, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.7462225033382529, + "rescued": 0, + "harmed": 58, + "missed_rescue": 1964, + "avoided_harm": 1113, + "selected_both_correct": 2787, + "selected_both_wrong": 0 + }, + { + "budget": 0.21, + "threshold": -1.4898060953782988e-06, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.7456602712769695, + "rescued": 0, + "harmed": 66, + "missed_rescue": 1964, + "avoided_harm": 1105, + "selected_both_correct": 2922, + "selected_both_wrong": 0 + }, + { + "budget": 0.22, + "threshold": -1.735574414851726e-06, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.7453088762386675, + "rescued": 0, + "harmed": 71, + "missed_rescue": 1964, + "avoided_harm": 1100, + "selected_both_correct": 3059, + "selected_both_wrong": 0 + }, + { + "budget": 0.23, + "threshold": -2.0881013824691763e-06, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.7446763651697238, + "rescued": 1, + "harmed": 81, + "missed_rescue": 1963, + "avoided_harm": 1090, + "selected_both_correct": 3190, + "selected_both_wrong": 0 + }, + { + "budget": 0.24, + "threshold": -2.5644872039265465e-06, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.7442546911237613, + "rescued": 1, + "harmed": 87, + "missed_rescue": 1963, + "avoided_harm": 1084, + "selected_both_correct": 3326, + "selected_both_wrong": 0 + }, + { + "budget": 0.25, + "threshold": -3.2399550491390983e-06, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.7437627380701385, + "rescued": 1, + "harmed": 94, + "missed_rescue": 1963, + "avoided_harm": 1077, + "selected_both_correct": 3462, + "selected_both_wrong": 0 + }, + { + "budget": 0.26, + "threshold": -4.2517635847616475e-06, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.7431302270011948, + "rescued": 1, + "harmed": 103, + "missed_rescue": 1963, + "avoided_harm": 1068, + "selected_both_correct": 3595, + "selected_both_wrong": 0 + }, + { + "budget": 0.27, + "threshold": -5.607922958006384e-06, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.7422868789092698, + "rescued": 1, + "harmed": 115, + "missed_rescue": 1963, + "avoided_harm": 1056, + "selected_both_correct": 3725, + "selected_both_wrong": 0 + }, + { + "budget": 0.28, + "threshold": -8.438555596512742e-06, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.7419354838709677, + "rescued": 1, + "harmed": 120, + "missed_rescue": 1963, + "avoided_harm": 1051, + "selected_both_correct": 3862, + "selected_both_wrong": 1 + }, + { + "budget": 0.29, + "threshold": -1.3145449884177651e-05, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.7411624147867032, + "rescued": 1, + "harmed": 131, + "missed_rescue": 1963, + "avoided_harm": 1040, + "selected_both_correct": 3993, + "selected_both_wrong": 1 + }, + { + "budget": 0.3, + "threshold": -2.076632881653495e-05, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.7404596247100991, + "rescued": 2, + "harmed": 142, + "missed_rescue": 1962, + "avoided_harm": 1029, + "selected_both_correct": 4123, + "selected_both_wrong": 1 + }, + { + "budget": 0.31, + "threshold": -4.0114435250870883e-05, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.7394757186028533, + "rescued": 2, + "harmed": 156, + "missed_rescue": 1962, + "avoided_harm": 1015, + "selected_both_correct": 4251, + "selected_both_wrong": 1 + }, + { + "budget": 0.32, + "threshold": -7.415730942739174e-05, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.7388432075339096, + "rescued": 2, + "harmed": 165, + "missed_rescue": 1962, + "avoided_harm": 1006, + "selected_both_correct": 4385, + "selected_both_wrong": 1 + }, + { + "budget": 0.33, + "threshold": -0.00015223574882838875, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.7377890224190035, + "rescued": 2, + "harmed": 180, + "missed_rescue": 1962, + "avoided_harm": 991, + "selected_both_correct": 4512, + "selected_both_wrong": 1 + }, + { + "budget": 0.34, + "threshold": -0.00032258269493468106, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.7361023262351536, + "rescued": 2, + "harmed": 204, + "missed_rescue": 1962, + "avoided_harm": 967, + "selected_both_correct": 4629, + "selected_both_wrong": 2 + }, + { + "budget": 0.35, + "threshold": -0.0006007538759149611, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.7348373040972661, + "rescued": 2, + "harmed": 222, + "missed_rescue": 1962, + "avoided_harm": 949, + "selected_both_correct": 4754, + "selected_both_wrong": 2 + }, + { + "budget": 0.36, + "threshold": -0.0011453087208792567, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.7337128399746996, + "rescued": 2, + "harmed": 238, + "missed_rescue": 1962, + "avoided_harm": 933, + "selected_both_correct": 4880, + "selected_both_wrong": 2 + }, + { + "budget": 0.37, + "threshold": -0.002608026610687375, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.732588375852133, + "rescued": 2, + "harmed": 254, + "missed_rescue": 1962, + "avoided_harm": 917, + "selected_both_correct": 5006, + "selected_both_wrong": 2 + }, + { + "budget": 0.38, + "threshold": -0.006219128146767616, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.7311827956989247, + "rescued": 2, + "harmed": 274, + "missed_rescue": 1962, + "avoided_harm": 897, + "selected_both_correct": 5128, + "selected_both_wrong": 3 + }, + { + "budget": 0.39, + "threshold": -0.01280172262340784, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.7297069365380561, + "rescued": 2, + "harmed": 295, + "missed_rescue": 1962, + "avoided_harm": 876, + "selected_both_correct": 5248, + "selected_both_wrong": 4 + }, + { + "budget": 0.4, + "threshold": -0.03794781118631363, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.7282310773771874, + "rescued": 3, + "harmed": 317, + "missed_rescue": 1961, + "avoided_harm": 854, + "selected_both_correct": 5365, + "selected_both_wrong": 6 + }, + { + "budget": 0.41, + "threshold": -0.08479184657335281, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.7266849392086584, + "rescued": 3, + "harmed": 339, + "missed_rescue": 1961, + "avoided_harm": 832, + "selected_both_correct": 5484, + "selected_both_wrong": 7 + }, + { + "budget": 0.42, + "threshold": -0.14941807091236115, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.7249279640171481, + "rescued": 7, + "harmed": 368, + "missed_rescue": 1957, + "avoided_harm": 803, + "selected_both_correct": 5590, + "selected_both_wrong": 11 + }, + { + "budget": 0.43, + "threshold": -0.2939925789833069, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.7233115468409586, + "rescued": 12, + "harmed": 396, + "missed_rescue": 1952, + "avoided_harm": 775, + "selected_both_correct": 5696, + "selected_both_wrong": 14 + }, + { + "budget": 0.44, + "threshold": -0.4386065602302551, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.7212734556188066, + "rescued": 15, + "harmed": 428, + "missed_rescue": 1949, + "avoided_harm": 743, + "selected_both_correct": 5798, + "selected_both_wrong": 19 + }, + { + "budget": 0.45, + "threshold": -0.5420411825180054, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.7190245273736735, + "rescued": 16, + "harmed": 461, + "missed_rescue": 1948, + "avoided_harm": 710, + "selected_both_correct": 5901, + "selected_both_wrong": 25 + }, + { + "budget": 0.46, + "threshold": -0.6230618953704834, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.7173378311898236, + "rescued": 21, + "harmed": 490, + "missed_rescue": 1943, + "avoided_harm": 681, + "selected_both_correct": 6008, + "selected_both_wrong": 26 + }, + { + "budget": 0.47, + "threshold": -0.6914289593696594, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.7155808559983133, + "rescued": 27, + "harmed": 521, + "missed_rescue": 1937, + "avoided_harm": 650, + "selected_both_correct": 6107, + "selected_both_wrong": 32 + }, + { + "budget": 0.48, + "threshold": -0.7387455701828003, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.7206409445498629, + "rescued": 105, + "harmed": 527, + "missed_rescue": 1859, + "avoided_harm": 644, + "selected_both_correct": 6125, + "selected_both_wrong": 72 + }, + { + "budget": 0.49, + "threshold": -0.7823225855827332, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.7280202403542062, + "rescued": 210, + "harmed": 527, + "missed_rescue": 1754, + "avoided_harm": 644, + "selected_both_correct": 6125, + "selected_both_wrong": 110 + }, + { + "budget": 0.5, + "threshold": -0.8206579089164734, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.7346264670742849, + "rescued": 305, + "harmed": 528, + "missed_rescue": 1659, + "avoided_harm": 643, + "selected_both_correct": 6125, + "selected_both_wrong": 156 + }, + { + "budget": 0.51, + "threshold": -0.8575630187988281, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.7414435308173448, + "rescued": 403, + "harmed": 529, + "missed_rescue": 1561, + "avoided_harm": 642, + "selected_both_correct": 6125, + "selected_both_wrong": 199 + }, + { + "budget": 0.52, + "threshold": -0.8823798894882202, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.7478389205144423, + "rescued": 495, + "harmed": 530, + "missed_rescue": 1469, + "avoided_harm": 641, + "selected_both_correct": 6125, + "selected_both_wrong": 249 + }, + { + "budget": 0.53, + "threshold": -0.9040287733078003, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.754093752196219, + "rescued": 585, + "harmed": 531, + "missed_rescue": 1379, + "avoided_harm": 640, + "selected_both_correct": 6125, + "selected_both_wrong": 300 + }, + { + "budget": 0.54, + "threshold": -0.9208687543869019, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.7602783048703352, + "rescued": 673, + "harmed": 531, + "missed_rescue": 1291, + "avoided_harm": 640, + "selected_both_correct": 6127, + "selected_both_wrong": 352 + }, + { + "budget": 0.55, + "threshold": -0.9344452619552612, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.7662520205214702, + "rescued": 759, + "harmed": 532, + "missed_rescue": 1205, + "avoided_harm": 639, + "selected_both_correct": 6129, + "selected_both_wrong": 405 + }, + { + "budget": 0.56, + "threshold": -0.9448322653770447, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.7718743411343032, + "rescued": 844, + "harmed": 537, + "missed_rescue": 1120, + "avoided_harm": 634, + "selected_both_correct": 6130, + "selected_both_wrong": 457 + }, + { + "budget": 0.57, + "threshold": -0.9531905055046082, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.7774263827394757, + "rescued": 924, + "harmed": 538, + "missed_rescue": 1040, + "avoided_harm": 633, + "selected_both_correct": 6133, + "selected_both_wrong": 515 + }, + { + "budget": 0.58, + "threshold": -0.9603281617164612, + "operator": "gt", + "actual_fraction": 0.5798720922060581, + "metric": 0.7818539602220816, + "rescued": 991, + "harmed": 542, + "missed_rescue": 973, + "avoided_harm": 629, + "selected_both_correct": 6135, + "selected_both_wrong": 583 + }, + { + "budget": 0.59, + "threshold": -0.9670801758766174, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.7864220957200084, + "rescued": 1061, + "harmed": 547, + "missed_rescue": 903, + "avoided_harm": 624, + "selected_both_correct": 6139, + "selected_both_wrong": 648 + }, + { + "budget": 0.6, + "threshold": -0.9724565148353577, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.7909902312179352, + "rescued": 1130, + "harmed": 551, + "missed_rescue": 834, + "avoided_harm": 620, + "selected_both_correct": 6146, + "selected_both_wrong": 710 + }, + { + "budget": 0.61, + "threshold": -0.9766654968261719, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.7954880877082016, + "rescued": 1200, + "harmed": 557, + "missed_rescue": 764, + "avoided_harm": 614, + "selected_both_correct": 6151, + "selected_both_wrong": 771 + }, + { + "budget": 0.62, + "threshold": -0.9801702499389648, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.7996345491601659, + "rescued": 1263, + "harmed": 561, + "missed_rescue": 701, + "avoided_harm": 610, + "selected_both_correct": 6159, + "selected_both_wrong": 838 + }, + { + "budget": 0.63, + "threshold": -0.9831460118293762, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.8028673835125448, + "rescued": 1317, + "harmed": 569, + "missed_rescue": 647, + "avoided_harm": 602, + "selected_both_correct": 6171, + "selected_both_wrong": 907 + }, + { + "budget": 0.64, + "threshold": -0.985751748085022, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.8064516129032258, + "rescued": 1380, + "harmed": 581, + "missed_rescue": 584, + "avoided_harm": 590, + "selected_both_correct": 6186, + "selected_both_wrong": 959 + }, + { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.809895284278586, + "rescued": 1434, + "harmed": 586, + "missed_rescue": 530, + "avoided_harm": 585, + "selected_both_correct": 6211, + "selected_both_wrong": 1017 + }, + { + "budget": 0.66, + "threshold": -0.9895955324172974, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.8123550495467003, + "rescued": 1479, + "harmed": 596, + "missed_rescue": 485, + "avoided_harm": 575, + "selected_both_correct": 6244, + "selected_both_wrong": 1072 + }, + { + "budget": 0.67, + "threshold": -0.9913432002067566, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.8149553728301356, + "rescued": 1525, + "harmed": 605, + "missed_rescue": 439, + "avoided_harm": 566, + "selected_both_correct": 6278, + "selected_both_wrong": 1125 + }, + { + "budget": 0.68, + "threshold": -0.9925730228424072, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.8169231850446271, + "rescued": 1569, + "harmed": 621, + "missed_rescue": 395, + "avoided_harm": 550, + "selected_both_correct": 6318, + "selected_both_wrong": 1167 + }, + { + "budget": 0.69, + "threshold": -0.9935283660888672, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.818609881228477, + "rescued": 1605, + "harmed": 633, + "missed_rescue": 359, + "avoided_harm": 538, + "selected_both_correct": 6376, + "selected_both_wrong": 1204 + }, + { + "budget": 0.7, + "threshold": -0.99436354637146, + "operator": "gt", + "actual_fraction": 0.6999086372900415, + "metric": 0.8196640663433832, + "rescued": 1636, + "harmed": 649, + "missed_rescue": 328, + "avoided_harm": 522, + "selected_both_correct": 6439, + "selected_both_wrong": 1235 + }, + { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.8201560193970061, + "rescued": 1660, + "harmed": 666, + "missed_rescue": 304, + "avoided_harm": 505, + "selected_both_correct": 6505, + "selected_both_wrong": 1271 + }, + { + "budget": 0.72, + "threshold": -0.9955723881721497, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.8200154613816852, + "rescued": 1686, + "harmed": 694, + "missed_rescue": 278, + "avoided_harm": 477, + "selected_both_correct": 6565, + "selected_both_wrong": 1299 + }, + { + "budget": 0.73, + "threshold": -0.9960037469863892, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.8199451823740249, + "rescued": 1703, + "harmed": 712, + "missed_rescue": 261, + "avoided_harm": 459, + "selected_both_correct": 6646, + "selected_both_wrong": 1326 + }, + { + "budget": 0.74, + "threshold": -0.9964693188667297, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.8200154613816852, + "rescued": 1722, + "harmed": 730, + "missed_rescue": 242, + "avoided_harm": 441, + "selected_both_correct": 6725, + "selected_both_wrong": 1352 + }, + { + "budget": 0.75, + "threshold": -0.9968041777610779, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.8198749033663645, + "rescued": 1742, + "harmed": 752, + "missed_rescue": 222, + "avoided_harm": 419, + "selected_both_correct": 6808, + "selected_both_wrong": 1369 + }, + { + "budget": 0.76, + "threshold": -0.9971042275428772, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.8200154613816852, + "rescued": 1763, + "harmed": 771, + "missed_rescue": 201, + "avoided_harm": 400, + "selected_both_correct": 6891, + "selected_both_wrong": 1389 + }, + { + "budget": 0.77, + "threshold": -0.9974007606506348, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.8195235083280624, + "rescued": 1784, + "harmed": 799, + "missed_rescue": 180, + "avoided_harm": 372, + "selected_both_correct": 6966, + "selected_both_wrong": 1407 + }, + { + "budget": 0.78, + "threshold": -0.9976577162742615, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.8191018342820999, + "rescued": 1800, + "harmed": 821, + "missed_rescue": 164, + "avoided_harm": 350, + "selected_both_correct": 7050, + "selected_both_wrong": 1427 + }, + { + "budget": 0.79, + "threshold": -0.9978766441345215, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.8192423922974208, + "rescued": 1818, + "harmed": 837, + "missed_rescue": 146, + "avoided_harm": 334, + "selected_both_correct": 7143, + "selected_both_wrong": 1442 + }, + { + "budget": 0.8, + "threshold": -0.9980920553207397, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.8190315552744395, + "rescued": 1834, + "harmed": 856, + "missed_rescue": 130, + "avoided_harm": 315, + "selected_both_correct": 7244, + "selected_both_wrong": 1449 + }, + { + "budget": 0.81, + "threshold": -0.9982843399047852, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.8190315552744395, + "rescued": 1849, + "harmed": 871, + "missed_rescue": 115, + "avoided_harm": 300, + "selected_both_correct": 7347, + "selected_both_wrong": 1458 + }, + { + "budget": 0.82, + "threshold": -0.9984900951385498, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.8187504392437979, + "rescued": 1859, + "harmed": 885, + "missed_rescue": 105, + "avoided_harm": 286, + "selected_both_correct": 7444, + "selected_both_wrong": 1479 + }, + { + "budget": 0.83, + "threshold": -0.9986395835876465, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.818609881228477, + "rescued": 1874, + "harmed": 902, + "missed_rescue": 90, + "avoided_harm": 269, + "selected_both_correct": 7541, + "selected_both_wrong": 1493 + }, + { + "budget": 0.84, + "threshold": -0.9987816214561462, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.8176962541288917, + "rescued": 1880, + "harmed": 921, + "missed_rescue": 84, + "avoided_harm": 250, + "selected_both_correct": 7643, + "selected_both_wrong": 1508 + }, + { + "budget": 0.85, + "threshold": -0.9989162087440491, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.8175556961135709, + "rescued": 1889, + "harmed": 932, + "missed_rescue": 75, + "avoided_harm": 239, + "selected_both_correct": 7754, + "selected_both_wrong": 1519 + }, + { + "budget": 0.86, + "threshold": -0.9990254044532776, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.8165717900063251, + "rescued": 1895, + "harmed": 952, + "missed_rescue": 69, + "avoided_harm": 219, + "selected_both_correct": 7864, + "selected_both_wrong": 1525 + }, + { + "budget": 0.87, + "threshold": -0.9991487264633179, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.8160798369527023, + "rescued": 1907, + "harmed": 971, + "missed_rescue": 57, + "avoided_harm": 200, + "selected_both_correct": 7969, + "selected_both_wrong": 1532 + }, + { + "budget": 0.88, + "threshold": -0.9992602467536926, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.8155878838990793, + "rescued": 1914, + "harmed": 985, + "missed_rescue": 50, + "avoided_harm": 186, + "selected_both_correct": 8082, + "selected_both_wrong": 1540 + }, + { + "budget": 0.89, + "threshold": -0.999350905418396, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.8148148148148148, + "rescued": 1919, + "harmed": 1001, + "missed_rescue": 45, + "avoided_harm": 170, + "selected_both_correct": 8192, + "selected_both_wrong": 1551 + }, + { + "budget": 0.9, + "threshold": -0.999435544013977, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.8144634197765127, + "rescued": 1927, + "harmed": 1014, + "missed_rescue": 37, + "avoided_harm": 157, + "selected_both_correct": 8308, + "selected_both_wrong": 1557 + }, + { + "budget": 0.91, + "threshold": -0.9995256066322327, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.8129172816079837, + "rescued": 1932, + "harmed": 1041, + "missed_rescue": 32, + "avoided_harm": 130, + "selected_both_correct": 8410, + "selected_both_wrong": 1565 + }, + { + "budget": 0.92, + "threshold": -0.9995898604393005, + "operator": "ge", + "actual_fraction": 0.9199522102747909, + "metric": 0.8123550495467003, + "rescued": 1937, + "harmed": 1054, + "missed_rescue": 27, + "avoided_harm": 117, + "selected_both_correct": 8530, + "selected_both_wrong": 1569 + }, + { + "budget": 0.93, + "threshold": -0.9996523857116699, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.8113711434394546, + "rescued": 1942, + "harmed": 1073, + "missed_rescue": 22, + "avoided_harm": 98, + "selected_both_correct": 8647, + "selected_both_wrong": 1570 + }, + { + "budget": 0.94, + "threshold": -0.9997095465660095, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.8102466793168881, + "rescued": 1945, + "harmed": 1092, + "missed_rescue": 19, + "avoided_harm": 79, + "selected_both_correct": 8764, + "selected_both_wrong": 1574 + }, + { + "budget": 0.95, + "threshold": -0.9997634887695312, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.8087708201560194, + "rescued": 1948, + "harmed": 1116, + "missed_rescue": 16, + "avoided_harm": 55, + "selected_both_correct": 8876, + "selected_both_wrong": 1577 + }, + { + "budget": 0.96, + "threshold": -0.9998065829277039, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.8079977510717549, + "rescued": 1951, + "harmed": 1130, + "missed_rescue": 13, + "avoided_harm": 41, + "selected_both_correct": 8998, + "selected_both_wrong": 1580 + }, + { + "budget": 0.97, + "threshold": -0.9998431205749512, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.8075057980181319, + "rescued": 1956, + "harmed": 1142, + "missed_rescue": 8, + "avoided_harm": 29, + "selected_both_correct": 9120, + "selected_both_wrong": 1584 + }, + { + "budget": 0.98, + "threshold": -0.9998764991760254, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.8070138449645091, + "rescued": 1961, + "harmed": 1154, + "missed_rescue": 3, + "avoided_harm": 17, + "selected_both_correct": 9242, + "selected_both_wrong": 1587 + }, + { + "budget": 0.99, + "threshold": -0.9999123811721802, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.806311054887905, + "rescued": 1962, + "harmed": 1165, + "missed_rescue": 2, + "avoided_harm": 6, + "selected_both_correct": 9370, + "selected_both_wrong": 1589 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8060299388572634, + "rescued": 1964, + "harmed": 1171, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9505, + "selected_both_wrong": 1589 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.6695119530385957, + "rescued": 0, + "harmed": 0, + "missed_rescue": 5423, + "avoided_harm": 2771, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -2.0005727208172175e-07, + "operator": "ge", + "actual_fraction": 0.002467175226157729, + "metric": 0.6686895612965431, + "rescued": 0, + "harmed": 29, + "missed_rescue": 5423, + "avoided_harm": 2742, + "selected_both_correct": 58, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -2.4156264544217265e-07, + "operator": "ge", + "actual_fraction": 0.004849275444516916, + "metric": 0.6680373195700876, + "rescued": 0, + "harmed": 52, + "missed_rescue": 5423, + "avoided_harm": 2719, + "selected_both_correct": 119, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -2.7729177531909954e-07, + "operator": "ge", + "actual_fraction": 0.007600034030003119, + "metric": 0.6675835861951621, + "rescued": 0, + "harmed": 68, + "missed_rescue": 5423, + "avoided_harm": 2703, + "selected_both_correct": 199, + "selected_both_wrong": 1 + }, + { + "budget": 0.04, + "threshold": -3.064858447032748e-07, + "operator": "ge", + "actual_fraction": 0.009811984232765221, + "metric": 0.6671014944843037, + "rescued": 0, + "harmed": 85, + "missed_rescue": 5423, + "avoided_harm": 2686, + "selected_both_correct": 260, + "selected_both_wrong": 1 + }, + { + "budget": 0.05, + "threshold": -3.3848826319626824e-07, + "operator": "ge", + "actual_fraction": 0.012534384482318578, + "metric": 0.6666194027734452, + "rescued": 0, + "harmed": 102, + "missed_rescue": 5423, + "avoided_harm": 2669, + "selected_both_correct": 338, + "selected_both_wrong": 2 + }, + { + "budget": 0.06, + "threshold": -3.668622525765386e-07, + "operator": "ge", + "actual_fraction": 0.01607917647392451, + "metric": 0.6662223860703854, + "rescued": 0, + "harmed": 116, + "missed_rescue": 5423, + "avoided_harm": 2655, + "selected_both_correct": 449, + "selected_both_wrong": 2 + }, + { + "budget": 0.07, + "threshold": -3.98026287484754e-07, + "operator": "ge", + "actual_fraction": 0.019709043473328986, + "metric": 0.6658253693673255, + "rescued": 0, + "harmed": 130, + "missed_rescue": 5423, + "avoided_harm": 2641, + "selected_both_correct": 562, + "selected_both_wrong": 3 + }, + { + "budget": 0.08, + "threshold": -4.302527258914779e-07, + "operator": "ge", + "actual_fraction": 0.02353741882426339, + "metric": 0.6655985026798628, + "rescued": 0, + "harmed": 138, + "missed_rescue": 5423, + "avoided_harm": 2633, + "selected_both_correct": 689, + "selected_both_wrong": 3 + }, + { + "budget": 0.09, + "threshold": -4.7191912244670675e-07, + "operator": "ge", + "actual_fraction": 0.028868785979638715, + "metric": 0.6651164109690043, + "rescued": 0, + "harmed": 155, + "missed_rescue": 5423, + "avoided_harm": 2616, + "selected_both_correct": 859, + "selected_both_wrong": 4 + }, + { + "budget": 0.1, + "threshold": -5.115426802149159e-07, + "operator": "ge", + "actual_fraction": 0.0345971698380739, + "metric": 0.6647761109378102, + "rescued": 0, + "harmed": 167, + "missed_rescue": 5423, + "avoided_harm": 2604, + "selected_both_correct": 1045, + "selected_both_wrong": 8 + }, + { + "budget": 0.11, + "threshold": -5.534793672268279e-07, + "operator": "ge", + "actual_fraction": 0.03981510364971783, + "metric": 0.664435810906616, + "rescued": 0, + "harmed": 179, + "missed_rescue": 5423, + "avoided_harm": 2592, + "selected_both_correct": 1215, + "selected_both_wrong": 10 + }, + { + "budget": 0.12, + "threshold": -5.941800509390305e-07, + "operator": "ge", + "actual_fraction": 0.045855429203414344, + "metric": 0.664067152539489, + "rescued": 0, + "harmed": 192, + "missed_rescue": 5423, + "avoided_harm": 2579, + "selected_both_correct": 1414, + "selected_both_wrong": 11 + }, + { + "budget": 0.13, + "threshold": -6.464504735959054e-07, + "operator": "ge", + "actual_fraction": 0.05317187987408899, + "metric": 0.6637552108442276, + "rescued": 1, + "harmed": 204, + "missed_rescue": 5422, + "avoided_harm": 2567, + "selected_both_correct": 1658, + "selected_both_wrong": 12 + }, + { + "budget": 0.14, + "threshold": -6.999837864896108e-07, + "operator": "ge", + "actual_fraction": 0.05960922213084536, + "metric": 0.6636134191645634, + "rescued": 1, + "harmed": 209, + "missed_rescue": 5422, + "avoided_harm": 2562, + "selected_both_correct": 1878, + "selected_both_wrong": 14 + }, + { + "budget": 0.15, + "threshold": -7.663418841730163e-07, + "operator": "ge", + "actual_fraction": 0.0675211978561098, + "metric": 0.6632164024615036, + "rescued": 2, + "harmed": 224, + "missed_rescue": 5421, + "avoided_harm": 2547, + "selected_both_correct": 2128, + "selected_both_wrong": 27 + }, + { + "budget": 0.16, + "threshold": -8.471382670904859e-07, + "operator": "ge", + "actual_fraction": 0.07551824858917279, + "metric": 0.6626208774069138, + "rescued": 2, + "harmed": 245, + "missed_rescue": 5421, + "avoided_harm": 2526, + "selected_both_correct": 2386, + "selected_both_wrong": 30 + }, + { + "budget": 0.17, + "threshold": -9.292756431023008e-07, + "operator": "ge", + "actual_fraction": 0.08212574086152624, + "metric": 0.6622522190397867, + "rescued": 3, + "harmed": 259, + "missed_rescue": 5420, + "avoided_harm": 2512, + "selected_both_correct": 2599, + "selected_both_wrong": 35 + }, + { + "budget": 0.18, + "threshold": -1.0245233852401725e-06, + "operator": "ge", + "actual_fraction": 0.08856308311828262, + "metric": 0.6619969940163911, + "rescued": 3, + "harmed": 268, + "missed_rescue": 5420, + "avoided_harm": 2503, + "selected_both_correct": 2812, + "selected_both_wrong": 40 + }, + { + "budget": 0.19, + "threshold": -1.1468601996966754e-06, + "operator": "ge", + "actual_fraction": 0.09480191702350906, + "metric": 0.6615149023055327, + "rescued": 3, + "harmed": 285, + "missed_rescue": 5420, + "avoided_harm": 2486, + "selected_both_correct": 3005, + "selected_both_wrong": 50 + }, + { + "budget": 0.2, + "threshold": -1.3018121762797819e-06, + "operator": "ge", + "actual_fraction": 0.10007656750701868, + "metric": 0.6612596772821371, + "rescued": 3, + "harmed": 294, + "missed_rescue": 5420, + "avoided_harm": 2477, + "selected_both_correct": 3174, + "selected_both_wrong": 58 + }, + { + "budget": 0.21, + "threshold": -1.4898060953782988e-06, + "operator": "ge", + "actual_fraction": 0.10554972634205825, + "metric": 0.6608343022431443, + "rescued": 3, + "harmed": 309, + "missed_rescue": 5420, + "avoided_harm": 2462, + "selected_both_correct": 3340, + "selected_both_wrong": 70 + }, + { + "budget": 0.22, + "threshold": -1.735574414851726e-06, + "operator": "ge", + "actual_fraction": 0.11005870175538099, + "metric": 0.6605507188838159, + "rescued": 3, + "harmed": 319, + "missed_rescue": 5420, + "avoided_harm": 2452, + "selected_both_correct": 3485, + "selected_both_wrong": 74 + }, + { + "budget": 0.23, + "threshold": -2.0881013824691763e-06, + "operator": "ge", + "actual_fraction": 0.11527663556702493, + "metric": 0.6600969855088903, + "rescued": 5, + "harmed": 337, + "missed_rescue": 5418, + "avoided_harm": 2434, + "selected_both_correct": 3630, + "selected_both_wrong": 93 + }, + { + "budget": 0.24, + "threshold": -2.5644872039265465e-06, + "operator": "ge", + "actual_fraction": 0.12029606102713893, + "metric": 0.6598417604854947, + "rescued": 8, + "harmed": 349, + "missed_rescue": 5415, + "avoided_harm": 2422, + "selected_both_correct": 3769, + "selected_both_wrong": 116 + }, + { + "budget": 0.25, + "threshold": -3.2399550491390983e-06, + "operator": "ge", + "actual_fraction": 0.12477667810452883, + "metric": 0.6594731021183677, + "rescued": 8, + "harmed": 362, + "missed_rescue": 5415, + "avoided_harm": 2409, + "selected_both_correct": 3894, + "selected_both_wrong": 136 + }, + { + "budget": 0.26, + "threshold": -4.2517635847616475e-06, + "operator": "ge", + "actual_fraction": 0.12914386183818732, + "metric": 0.6590193687434421, + "rescued": 8, + "harmed": 378, + "missed_rescue": 5415, + "avoided_harm": 2393, + "selected_both_correct": 4012, + "selected_both_wrong": 156 + }, + { + "budget": 0.27, + "threshold": -5.607922958006384e-06, + "operator": "ge", + "actual_fraction": 0.1322916371267334, + "metric": 0.658679068712248, + "rescued": 10, + "harmed": 392, + "missed_rescue": 5413, + "avoided_harm": 2379, + "selected_both_correct": 4092, + "selected_both_wrong": 171 + }, + { + "budget": 0.28, + "threshold": -8.438555596512742e-06, + "operator": "ge", + "actual_fraction": 0.13626180415733205, + "metric": 0.6582820520091881, + "rescued": 12, + "harmed": 408, + "missed_rescue": 5411, + "avoided_harm": 2363, + "selected_both_correct": 4192, + "selected_both_wrong": 193 + }, + { + "budget": 0.29, + "threshold": -1.3145449884177651e-05, + "operator": "ge", + "actual_fraction": 0.14023197118793068, + "metric": 0.657743243626464, + "rescued": 13, + "harmed": 428, + "missed_rescue": 5410, + "avoided_harm": 2343, + "selected_both_correct": 4285, + "selected_both_wrong": 219 + }, + { + "budget": 0.3, + "threshold": -2.076632881653495e-05, + "operator": "ge", + "actual_fraction": 0.14326631313274538, + "metric": 0.657374585259337, + "rescued": 16, + "harmed": 444, + "missed_rescue": 5407, + "avoided_harm": 2327, + "selected_both_correct": 4359, + "selected_both_wrong": 233 + }, + { + "budget": 0.31, + "threshold": -4.0114435250870883e-05, + "operator": "ge", + "actual_fraction": 0.14766185520233674, + "metric": 0.6569775685562771, + "rescued": 18, + "harmed": 460, + "missed_rescue": 5405, + "avoided_harm": 2311, + "selected_both_correct": 4464, + "selected_both_wrong": 265 + }, + { + "budget": 0.32, + "threshold": -7.415730942739174e-05, + "operator": "ge", + "actual_fraction": 0.15066783881121856, + "metric": 0.6567507018688143, + "rescued": 20, + "harmed": 470, + "missed_rescue": 5403, + "avoided_harm": 2301, + "selected_both_correct": 4545, + "selected_both_wrong": 278 + }, + { + "budget": 0.33, + "threshold": -0.00015223574882838875, + "operator": "ge", + "actual_fraction": 0.15452457249808582, + "metric": 0.6565238351813516, + "rescued": 27, + "harmed": 485, + "missed_rescue": 5396, + "avoided_harm": 2286, + "selected_both_correct": 4632, + "selected_both_wrong": 305 + }, + { + "budget": 0.34, + "threshold": -0.00032258269493468106, + "operator": "ge", + "actual_fraction": 0.15806936448969175, + "metric": 0.6564104018376201, + "rescued": 39, + "harmed": 501, + "missed_rescue": 5384, + "avoided_harm": 2270, + "selected_both_correct": 4705, + "selected_both_wrong": 329 + }, + { + "budget": 0.35, + "threshold": -0.0006007538759149611, + "operator": "ge", + "actual_fraction": 0.16201117318435754, + "metric": 0.6562686101579559, + "rescued": 46, + "harmed": 513, + "missed_rescue": 5377, + "avoided_harm": 2258, + "selected_both_correct": 4777, + "selected_both_wrong": 377 + }, + { + "budget": 0.36, + "threshold": -0.0011453087208792567, + "operator": "ge", + "actual_fraction": 0.1657544735274934, + "metric": 0.6558999517908289, + "rescued": 48, + "harmed": 528, + "missed_rescue": 5375, + "avoided_harm": 2243, + "selected_both_correct": 4841, + "selected_both_wrong": 428 + }, + { + "budget": 0.37, + "threshold": -0.002608026610687375, + "operator": "ge", + "actual_fraction": 0.17012165726115192, + "metric": 0.6558715934548961, + "rescued": 61, + "harmed": 542, + "missed_rescue": 5362, + "avoided_harm": 2229, + "selected_both_correct": 4937, + "selected_both_wrong": 459 + }, + { + "budget": 0.38, + "threshold": -0.006219128146767616, + "operator": "ge", + "actual_fraction": 0.17536794940872868, + "metric": 0.6555596517596347, + "rescued": 79, + "harmed": 571, + "missed_rescue": 5344, + "avoided_harm": 2200, + "selected_both_correct": 5018, + "selected_both_wrong": 516 + }, + { + "budget": 0.39, + "threshold": -0.01280172262340784, + "operator": "ge", + "actual_fraction": 0.17965005813458867, + "metric": 0.6553044267362391, + "rescued": 92, + "harmed": 593, + "missed_rescue": 5331, + "avoided_harm": 2178, + "selected_both_correct": 5091, + "selected_both_wrong": 559 + }, + { + "budget": 0.4, + "threshold": -0.03794781118631363, + "operator": "ge", + "actual_fraction": 0.18401724186824717, + "metric": 0.6552193517284406, + "rescued": 107, + "harmed": 611, + "missed_rescue": 5316, + "avoided_harm": 2160, + "selected_both_correct": 5145, + "selected_both_wrong": 626 + }, + { + "budget": 0.41, + "threshold": -0.08479184657335281, + "operator": "ge", + "actual_fraction": 0.18858293395343562, + "metric": 0.6552760684003063, + "rescued": 127, + "harmed": 629, + "missed_rescue": 5296, + "avoided_harm": 2142, + "selected_both_correct": 5207, + "selected_both_wrong": 687 + }, + { + "budget": 0.42, + "threshold": -0.14941807091236115, + "operator": "ge", + "actual_fraction": 0.19238295096843716, + "metric": 0.6554745767518362, + "rescued": 144, + "harmed": 639, + "missed_rescue": 5279, + "avoided_harm": 2132, + "selected_both_correct": 5277, + "selected_both_wrong": 724 + }, + { + "budget": 0.43, + "threshold": -0.2939925789833069, + "operator": "ge", + "actual_fraction": 0.19689192638175992, + "metric": 0.655502935087769, + "rescued": 168, + "harmed": 662, + "missed_rescue": 5255, + "avoided_harm": 2109, + "selected_both_correct": 5344, + "selected_both_wrong": 769 + }, + { + "budget": 0.44, + "threshold": -0.4386065602302551, + "operator": "ge", + "actual_fraction": 0.20182627683407536, + "metric": 0.6555880100955676, + "rescued": 199, + "harmed": 690, + "missed_rescue": 5224, + "avoided_harm": 2081, + "selected_both_correct": 5429, + "selected_both_wrong": 799 + }, + { + "budget": 0.45, + "threshold": -0.5420411825180054, + "operator": "ge", + "actual_fraction": 0.20718600232538353, + "metric": 0.6559283101267618, + "rescued": 240, + "harmed": 719, + "missed_rescue": 5183, + "avoided_harm": 2052, + "selected_both_correct": 5506, + "selected_both_wrong": 841 + }, + { + "budget": 0.46, + "threshold": -0.6230618953704834, + "operator": "ge", + "actual_fraction": 0.21189348609023623, + "metric": 0.656070101806426, + "rescued": 280, + "harmed": 754, + "missed_rescue": 5143, + "avoided_harm": 2017, + "selected_both_correct": 5554, + "selected_both_wrong": 884 + }, + { + "budget": 0.47, + "threshold": -0.6914289593696594, + "operator": "ge", + "actual_fraction": 0.217139778237813, + "metric": 0.6567790602047472, + "rescued": 329, + "harmed": 778, + "missed_rescue": 5094, + "avoided_harm": 1993, + "selected_both_correct": 5614, + "selected_both_wrong": 936 + }, + { + "budget": 0.48, + "threshold": -0.7387455701828003, + "operator": "ge", + "actual_fraction": 0.22198905368232993, + "metric": 0.6584238436888523, + "rescued": 399, + "harmed": 790, + "missed_rescue": 5024, + "avoided_harm": 1981, + "selected_both_correct": 5648, + "selected_both_wrong": 991 + }, + { + "budget": 0.49, + "threshold": -0.7823225855827332, + "operator": "ge", + "actual_fraction": 0.22763236253296656, + "metric": 0.6604372855400845, + "rescued": 483, + "harmed": 803, + "missed_rescue": 4940, + "avoided_harm": 1968, + "selected_both_correct": 5689, + "selected_both_wrong": 1052 + }, + { + "budget": 0.5, + "threshold": -0.8206579089164734, + "operator": "ge", + "actual_fraction": 0.2345234381646485, + "metric": 0.662762669086578, + "rescued": 579, + "harmed": 817, + "missed_rescue": 4844, + "avoided_harm": 1954, + "selected_both_correct": 5735, + "selected_both_wrong": 1139 + }, + { + "budget": 0.51, + "threshold": -0.8575630187988281, + "operator": "ge", + "actual_fraction": 0.24382497235062248, + "metric": 0.6654567110001985, + "rescued": 702, + "harmed": 845, + "missed_rescue": 4721, + "avoided_harm": 1926, + "selected_both_correct": 5794, + "selected_both_wrong": 1257 + }, + { + "budget": 0.52, + "threshold": -0.8823798894882202, + "operator": "ge", + "actual_fraction": 0.252644414825738, + "metric": 0.6678955278904234, + "rescued": 813, + "harmed": 870, + "missed_rescue": 4610, + "avoided_harm": 1901, + "selected_both_correct": 5842, + "selected_both_wrong": 1384 + }, + { + "budget": 0.53, + "threshold": -0.9040287733078003, + "operator": "ge", + "actual_fraction": 0.26401610753480986, + "metric": 0.6703627031165811, + "rescued": 937, + "harmed": 907, + "missed_rescue": 4486, + "avoided_harm": 1864, + "selected_both_correct": 5946, + "selected_both_wrong": 1520 + }, + { + "budget": 0.54, + "threshold": -0.9208687543869019, + "operator": "ge", + "actual_fraction": 0.2760116836344043, + "metric": 0.6731134617020673, + "rescued": 1067, + "harmed": 940, + "missed_rescue": 4356, + "avoided_harm": 1831, + "selected_both_correct": 6068, + "selected_both_wrong": 1658 + }, + { + "budget": 0.55, + "threshold": -0.9344452619552612, + "operator": "ge", + "actual_fraction": 0.2893401015228426, + "metric": 0.6758358619516207, + "rescued": 1197, + "harmed": 974, + "missed_rescue": 4226, + "avoided_harm": 1797, + "selected_both_correct": 6218, + "selected_both_wrong": 1814 + }, + { + "budget": 0.56, + "threshold": -0.9448322653770447, + "operator": "ge", + "actual_fraction": 0.3036043444970649, + "metric": 0.679295578935428, + "rescued": 1352, + "harmed": 1007, + "missed_rescue": 4071, + "avoided_harm": 1764, + "selected_both_correct": 6385, + "selected_both_wrong": 1962 + }, + { + "budget": 0.57, + "threshold": -0.9531905055046082, + "operator": "ge", + "actual_fraction": 0.3175850041119587, + "metric": 0.6822448458724442, + "rescued": 1489, + "harmed": 1040, + "missed_rescue": 3934, + "avoided_harm": 1731, + "selected_both_correct": 6573, + "selected_both_wrong": 2097 + }, + { + "budget": 0.58, + "threshold": -0.9603281617164612, + "operator": "gt", + "actual_fraction": 0.33462836400760004, + "metric": 0.6853926211609903, + "rescued": 1645, + "harmed": 1085, + "missed_rescue": 3778, + "avoided_harm": 1686, + "selected_both_correct": 6816, + "selected_both_wrong": 2254 + }, + { + "budget": 0.59, + "threshold": -0.9670801758766174, + "operator": "ge", + "actual_fraction": 0.35637920766809406, + "metric": 0.6889090548166633, + "rescued": 1818, + "harmed": 1134, + "missed_rescue": 3605, + "avoided_harm": 1637, + "selected_both_correct": 7175, + "selected_both_wrong": 2440 + }, + { + "budget": 0.6, + "threshold": -0.9724565148353577, + "operator": "ge", + "actual_fraction": 0.37968975980489467, + "metric": 0.6922553384567394, + "rescued": 1978, + "harmed": 1176, + "missed_rescue": 3445, + "avoided_harm": 1595, + "selected_both_correct": 7569, + "selected_both_wrong": 2666 + }, + { + "budget": 0.61, + "threshold": -0.9766654968261719, + "operator": "ge", + "actual_fraction": 0.4046450954258004, + "metric": 0.6949493803703599, + "rescued": 2126, + "harmed": 1229, + "missed_rescue": 3297, + "avoided_harm": 1542, + "selected_both_correct": 8013, + "selected_both_wrong": 2901 + }, + { + "budget": 0.62, + "threshold": -0.9801702499389648, + "operator": "ge", + "actual_fraction": 0.43141536454640844, + "metric": 0.6983807390182344, + "rescued": 2296, + "harmed": 1278, + "missed_rescue": 3127, + "avoided_harm": 1493, + "selected_both_correct": 8524, + "selected_both_wrong": 3115 + }, + { + "budget": 0.63, + "threshold": -0.9831460118293762, + "operator": "ge", + "actual_fraction": 0.4610214672603012, + "metric": 0.7015568726427134, + "rescued": 2479, + "harmed": 1349, + "missed_rescue": 2944, + "avoided_harm": 1422, + "selected_both_correct": 9071, + "selected_both_wrong": 3358 + }, + { + "budget": 0.64, + "threshold": -0.985751748085022, + "operator": "ge", + "actual_fraction": 0.4916484700677764, + "metric": 0.7053285313217821, + "rescued": 2674, + "harmed": 1411, + "missed_rescue": 2749, + "avoided_harm": 1360, + "selected_both_correct": 9690, + "selected_both_wrong": 3562 + }, + { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.5209426310864078, + "metric": 0.7086180982899923, + "rescued": 2849, + "harmed": 1470, + "missed_rescue": 2574, + "avoided_harm": 1301, + "selected_both_correct": 10321, + "selected_both_wrong": 3730 + }, + { + "budget": 0.66, + "threshold": -0.9895955324172974, + "operator": "ge", + "actual_fraction": 0.5529308340186598, + "metric": 0.71148229021921, + "rescued": 3014, + "harmed": 1534, + "missed_rescue": 2409, + "avoided_harm": 1237, + "selected_both_correct": 11019, + "selected_both_wrong": 3931 + }, + { + "budget": 0.67, + "threshold": -0.9913432002067566, + "operator": "ge", + "actual_fraction": 0.5888041289737118, + "metric": 0.7155091739216742, + "rescued": 3227, + "harmed": 1605, + "missed_rescue": 2196, + "avoided_harm": 1166, + "selected_both_correct": 11762, + "selected_both_wrong": 4169 + }, + { + "budget": 0.68, + "threshold": -0.9925730228424072, + "operator": "ge", + "actual_fraction": 0.6175027649377535, + "metric": 0.7182882908430933, + "rescued": 3404, + "harmed": 1684, + "missed_rescue": 2019, + "avoided_harm": 1087, + "selected_both_correct": 12326, + "selected_both_wrong": 4361 + }, + { + "budget": 0.69, + "threshold": -0.9935283660888672, + "operator": "ge", + "actual_fraction": 0.6421461588633979, + "metric": 0.7211241244363781, + "rescued": 3553, + "harmed": 1733, + "missed_rescue": 1870, + "avoided_harm": 1038, + "selected_both_correct": 12849, + "selected_both_wrong": 4509 + }, + { + "budget": 0.7, + "threshold": -0.99436354637146, + "operator": "gt", + "actual_fraction": 0.665910444375124, + "metric": 0.7233077163032073, + "rescued": 3694, + "harmed": 1797, + "missed_rescue": 1729, + "avoided_harm": 974, + "selected_both_correct": 13314, + "selected_both_wrong": 4677 + }, + { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.6838612710206166, + "metric": 0.7243286163967898, + "rescued": 3794, + "harmed": 1861, + "missed_rescue": 1629, + "avoided_harm": 910, + "selected_both_correct": 13678, + "selected_both_wrong": 4782 + }, + { + "budget": 0.72, + "threshold": -0.9955723881721497, + "operator": "ge", + "actual_fraction": 0.7029464311034228, + "metric": 0.7260017582168279, + "rescued": 3908, + "harmed": 1916, + "missed_rescue": 1515, + "avoided_harm": 855, + "selected_both_correct": 14058, + "selected_both_wrong": 4906 + }, + { + "budget": 0.73, + "threshold": -0.9960037469863892, + "operator": "ge", + "actual_fraction": 0.7180614241556306, + "metric": 0.7278450500524629, + "rescued": 4015, + "harmed": 1958, + "missed_rescue": 1408, + "avoided_harm": 813, + "selected_both_correct": 14354, + "selected_both_wrong": 4994 + }, + { + "budget": 0.74, + "threshold": -0.9964693188667297, + "operator": "ge", + "actual_fraction": 0.7352465757309361, + "metric": 0.7297450585599637, + "rescued": 4134, + "harmed": 2010, + "missed_rescue": 1289, + "avoided_harm": 761, + "selected_both_correct": 14689, + "selected_both_wrong": 5094 + }, + { + "budget": 0.75, + "threshold": -0.9968041777610779, + "operator": "ge", + "actual_fraction": 0.7478093185491875, + "metric": 0.7311629753566061, + "rescued": 4208, + "harmed": 2034, + "missed_rescue": 1215, + "avoided_harm": 737, + "selected_both_correct": 14961, + "selected_both_wrong": 5167 + }, + { + "budget": 0.76, + "threshold": -0.9971042275428772, + "operator": "ge", + "actual_fraction": 0.759804894648782, + "metric": 0.7328928338485098, + "rescued": 4299, + "harmed": 2064, + "missed_rescue": 1124, + "avoided_harm": 707, + "selected_both_correct": 15186, + "selected_both_wrong": 5244 + }, + { + "budget": 0.77, + "threshold": -0.9974007606506348, + "operator": "ge", + "actual_fraction": 0.7729064458497575, + "metric": 0.7342540339732865, + "rescued": 4385, + "harmed": 2102, + "missed_rescue": 1038, + "avoided_harm": 669, + "selected_both_correct": 15447, + "selected_both_wrong": 5321 + }, + { + "budget": 0.78, + "threshold": -0.9976577162742615, + "operator": "ge", + "actual_fraction": 0.7855259053398747, + "metric": 0.735274934066869, + "rescued": 4462, + "harmed": 2143, + "missed_rescue": 961, + "avoided_harm": 628, + "selected_both_correct": 15703, + "selected_both_wrong": 5392 + }, + { + "budget": 0.79, + "threshold": -0.9978766441345215, + "operator": "ge", + "actual_fraction": 0.7964722230099538, + "metric": 0.7362958341604514, + "rescued": 4527, + "harmed": 2172, + "missed_rescue": 896, + "avoided_harm": 599, + "selected_both_correct": 15920, + "selected_both_wrong": 5467 + }, + { + "budget": 0.8, + "threshold": -0.9980920553207397, + "operator": "ge", + "actual_fraction": 0.8082125740861527, + "metric": 0.7377704676289595, + "rescued": 4609, + "harmed": 2202, + "missed_rescue": 814, + "avoided_harm": 569, + "selected_both_correct": 16159, + "selected_both_wrong": 5530 + }, + { + "budget": 0.81, + "threshold": -0.9982843399047852, + "operator": "ge", + "actual_fraction": 0.8197544168108215, + "metric": 0.738422709355415, + "rescued": 4678, + "harmed": 2248, + "missed_rescue": 745, + "avoided_harm": 523, + "selected_both_correct": 16389, + "selected_both_wrong": 5592 + }, + { + "budget": 0.82, + "threshold": -0.9984900951385498, + "operator": "ge", + "actual_fraction": 0.8333380597226555, + "metric": 0.7389048010662734, + "rescued": 4746, + "harmed": 2299, + "missed_rescue": 677, + "avoided_harm": 472, + "selected_both_correct": 16674, + "selected_both_wrong": 5667 + }, + { + "budget": 0.83, + "threshold": -0.9986395835876465, + "operator": "ge", + "actual_fraction": 0.8426112355726966, + "metric": 0.7398689844879902, + "rescued": 4807, + "harmed": 2326, + "missed_rescue": 616, + "avoided_harm": 445, + "selected_both_correct": 16857, + "selected_both_wrong": 5723 + }, + { + "budget": 0.84, + "threshold": -0.9987816214561462, + "operator": "ge", + "actual_fraction": 0.8529903865241187, + "metric": 0.7403227178629158, + "rescued": 4859, + "harmed": 2362, + "missed_rescue": 564, + "avoided_harm": 409, + "selected_both_correct": 17089, + "selected_both_wrong": 5769 + }, + { + "budget": 0.85, + "threshold": -0.9989162087440491, + "operator": "ge", + "actual_fraction": 0.8635396874911381, + "metric": 0.7406913762300428, + "rescued": 4914, + "harmed": 2404, + "missed_rescue": 509, + "avoided_harm": 367, + "selected_both_correct": 17300, + "selected_both_wrong": 5833 + }, + { + "budget": 0.86, + "threshold": -0.9990254044532776, + "operator": "ge", + "actual_fraction": 0.8710262881774098, + "metric": 0.7413152596205654, + "rescued": 4962, + "harmed": 2430, + "missed_rescue": 461, + "avoided_harm": 341, + "selected_both_correct": 17455, + "selected_both_wrong": 5868 + }, + { + "budget": 0.87, + "threshold": -0.9991487264633179, + "operator": "ge", + "actual_fraction": 0.8809517057539064, + "metric": 0.7422227263704165, + "rescued": 5018, + "harmed": 2454, + "missed_rescue": 405, + "avoided_harm": 317, + "selected_both_correct": 17666, + "selected_both_wrong": 5927 + }, + { + "budget": 0.88, + "threshold": -0.9992602467536926, + "operator": "ge", + "actual_fraction": 0.8913024983693957, + "metric": 0.7430167597765364, + "rescued": 5074, + "harmed": 2482, + "missed_rescue": 349, + "avoided_harm": 289, + "selected_both_correct": 17892, + "selected_both_wrong": 5982 + }, + { + "budget": 0.89, + "threshold": -0.999350905418396, + "operator": "ge", + "actual_fraction": 0.900915974250631, + "metric": 0.743442134815529, + "rescued": 5120, + "harmed": 2513, + "missed_rescue": 303, + "avoided_harm": 258, + "selected_both_correct": 18112, + "selected_both_wrong": 6024 + }, + { + "budget": 0.9, + "threshold": -0.999435544013977, + "operator": "ge", + "actual_fraction": 0.910954825170859, + "metric": 0.7443212432294473, + "rescued": 5172, + "harmed": 2534, + "missed_rescue": 251, + "avoided_harm": 237, + "selected_both_correct": 18362, + "selected_both_wrong": 6055 + }, + { + "budget": 0.91, + "threshold": -0.9995256066322327, + "operator": "ge", + "actual_fraction": 0.921901142840938, + "metric": 0.7448033349403057, + "rescued": 5223, + "harmed": 2568, + "missed_rescue": 200, + "avoided_harm": 203, + "selected_both_correct": 18626, + "selected_both_wrong": 6092 + }, + { + "budget": 0.92, + "threshold": -0.9995898604393005, + "operator": "ge", + "actual_fraction": 0.9303235686129938, + "metric": 0.7450018432918356, + "rescued": 5253, + "harmed": 2591, + "missed_rescue": 170, + "avoided_harm": 180, + "selected_both_correct": 18846, + "selected_both_wrong": 6116 + }, + { + "budget": 0.93, + "threshold": -0.9996523857116699, + "operator": "ge", + "actual_fraction": 0.9393698777755721, + "metric": 0.7448884099481042, + "rescued": 5284, + "harmed": 2626, + "missed_rescue": 139, + "avoided_harm": 145, + "selected_both_correct": 19077, + "selected_both_wrong": 6138 + }, + { + "budget": 0.94, + "threshold": -0.9997095465660095, + "operator": "ge", + "actual_fraction": 0.9490117119927403, + "metric": 0.7448884099481042, + "rescued": 5314, + "harmed": 2656, + "missed_rescue": 109, + "avoided_harm": 115, + "selected_both_correct": 19329, + "selected_both_wrong": 6166 + }, + { + "budget": 0.95, + "threshold": -0.9997634887695312, + "operator": "ge", + "actual_fraction": 0.9587953378895726, + "metric": 0.7449734849559028, + "rescued": 5347, + "harmed": 2686, + "missed_rescue": 76, + "avoided_harm": 85, + "selected_both_correct": 19588, + "selected_both_wrong": 6189 + }, + { + "budget": 0.96, + "threshold": -0.9998065829277039, + "operator": "ge", + "actual_fraction": 0.9673595553412926, + "metric": 0.7452570683152313, + "rescued": 5375, + "harmed": 2704, + "missed_rescue": 48, + "avoided_harm": 67, + "selected_both_correct": 19828, + "selected_both_wrong": 6205 + }, + { + "budget": 0.97, + "threshold": -0.9998431205749512, + "operator": "ge", + "actual_fraction": 0.9746192893401016, + "metric": 0.745313784987097, + "rescued": 5398, + "harmed": 2725, + "missed_rescue": 25, + "avoided_harm": 46, + "selected_both_correct": 20033, + "selected_both_wrong": 6212 + }, + { + "budget": 0.98, + "threshold": -0.9998764991760254, + "operator": "ge", + "actual_fraction": 0.9830984317840229, + "metric": 0.7450585599637013, + "rescued": 5411, + "harmed": 2747, + "missed_rescue": 12, + "avoided_harm": 24, + "selected_both_correct": 20290, + "selected_both_wrong": 6219 + }, + { + "budget": 0.99, + "threshold": -0.9999123811721802, + "operator": "ge", + "actual_fraction": 0.9908118991577575, + "metric": 0.7448884099481042, + "rescued": 5419, + "harmed": 2761, + "missed_rescue": 4, + "avoided_harm": 10, + "selected_both_correct": 20535, + "selected_both_wrong": 6224 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.7447182599325072, + "rescued": 5423, + "harmed": 2771, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 20838, + "selected_both_wrong": 6231 + } + ] + }, + "learned": { + "selected_budget_index": 56, + "validation_area_0_25": 0.777812917281608, + "validation_point": { + "budget": 0.56, + "threshold": -5.435944080352783, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.8081383090870757, + "rescued": 1513, + "harmed": 690, + "missed_rescue": 451, + "avoided_harm": 481, + "selected_both_correct": 4599, + "selected_both_wrong": 1166 + }, + "test_point": { + "budget": 0.56, + "threshold": -5.435944080352783, + "operator": "ge", + "actual_fraction": 0.5355755324277571, + "metric": 0.7528287440093016, + "rescued": 4361, + "harmed": 1423, + "missed_rescue": 1062, + "avoided_harm": 1348, + "selected_both_correct": 8197, + "selected_both_wrong": 4905 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7502986857825568, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1964, + "avoided_harm": 1171, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 5.873929023742676, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.7559210063953897, + "rescued": 80, + "harmed": 0, + "missed_rescue": 1884, + "avoided_harm": 1171, + "selected_both_correct": 1, + "selected_both_wrong": 61 + }, + { + "budget": 0.02, + "threshold": 3.9572336673736572, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.7606999789162977, + "rescued": 149, + "harmed": 1, + "missed_rescue": 1815, + "avoided_harm": 1170, + "selected_both_correct": 3, + "selected_both_wrong": 131 + }, + { + "budget": 0.03, + "threshold": 2.499727725982666, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.765197835406564, + "rescued": 217, + "harmed": 5, + "missed_rescue": 1747, + "avoided_harm": 1166, + "selected_both_correct": 16, + "selected_both_wrong": 188 + }, + { + "budget": 0.04, + "threshold": 1.2223308086395264, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.7671656476210555, + "rescued": 258, + "harmed": 18, + "missed_rescue": 1706, + "avoided_harm": 1153, + "selected_both_correct": 51, + "selected_both_wrong": 242 + }, + { + "budget": 0.05, + "threshold": 0.36731216311454773, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.767587321667018, + "rescued": 283, + "harmed": 37, + "missed_rescue": 1681, + "avoided_harm": 1134, + "selected_both_correct": 113, + "selected_both_wrong": 278 + }, + { + "budget": 0.06, + "threshold": -0.14144867658615112, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.7663925785367911, + "rescued": 293, + "harmed": 64, + "missed_rescue": 1671, + "avoided_harm": 1107, + "selected_both_correct": 199, + "selected_both_wrong": 297 + }, + { + "budget": 0.07, + "threshold": -0.42117512226104736, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.7665331365521119, + "rescued": 320, + "harmed": 89, + "missed_rescue": 1644, + "avoided_harm": 1082, + "selected_both_correct": 259, + "selected_both_wrong": 328 + }, + { + "budget": 0.08, + "threshold": -0.6775990128517151, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.7686415067819242, + "rescued": 368, + "harmed": 107, + "missed_rescue": 1596, + "avoided_harm": 1064, + "selected_both_correct": 304, + "selected_both_wrong": 359 + }, + { + "budget": 0.09, + "threshold": -0.9111056327819824, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.7707498770117366, + "rescued": 412, + "harmed": 121, + "missed_rescue": 1552, + "avoided_harm": 1050, + "selected_both_correct": 343, + "selected_both_wrong": 404 + }, + { + "budget": 0.1, + "threshold": -1.1332147121429443, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.772858247241549, + "rescued": 454, + "harmed": 133, + "missed_rescue": 1510, + "avoided_harm": 1038, + "selected_both_correct": 393, + "selected_both_wrong": 442 + }, + { + "budget": 0.11, + "threshold": -1.3457791805267334, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.774404385410078, + "rescued": 491, + "harmed": 148, + "missed_rescue": 1473, + "avoided_harm": 1023, + "selected_both_correct": 446, + "selected_both_wrong": 480 + }, + { + "budget": 0.12, + "threshold": -1.5916669368743896, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.7769344296858528, + "rescued": 541, + "harmed": 162, + "missed_rescue": 1423, + "avoided_harm": 1009, + "selected_both_correct": 492, + "selected_both_wrong": 512 + }, + { + "budget": 0.13, + "threshold": -1.7836966514587402, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.7791130789233256, + "rescued": 584, + "harmed": 174, + "missed_rescue": 1380, + "avoided_harm": 997, + "selected_both_correct": 543, + "selected_both_wrong": 548 + }, + { + "budget": 0.14, + "threshold": -1.9795069694519043, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.781221449153138, + "rescued": 632, + "harmed": 192, + "missed_rescue": 1332, + "avoided_harm": 979, + "selected_both_correct": 597, + "selected_both_wrong": 571 + }, + { + "budget": 0.15, + "threshold": -2.1732606887817383, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.7838920514442336, + "rescued": 678, + "harmed": 200, + "missed_rescue": 1286, + "avoided_harm": 971, + "selected_both_correct": 654, + "selected_both_wrong": 602 + }, + { + "budget": 0.16, + "threshold": -2.3698818683624268, + "operator": "gt", + "actual_fraction": 0.15988474242743692, + "metric": 0.786351816712348, + "rescued": 724, + "harmed": 211, + "missed_rescue": 1240, + "avoided_harm": 960, + "selected_both_correct": 713, + "selected_both_wrong": 627 + }, + { + "budget": 0.17, + "threshold": -2.5250070095062256, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.7876168388502355, + "rescued": 756, + "harmed": 225, + "missed_rescue": 1208, + "avoided_harm": 946, + "selected_both_correct": 785, + "selected_both_wrong": 652 + }, + { + "budget": 0.18, + "threshold": -2.659658908843994, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.7886710239651417, + "rescued": 793, + "harmed": 247, + "missed_rescue": 1171, + "avoided_harm": 924, + "selected_both_correct": 845, + "selected_both_wrong": 676 + }, + { + "budget": 0.19, + "threshold": -2.7833452224731445, + "operator": "ge", + "actual_fraction": 0.18996415770609318, + "metric": 0.7901468831260102, + "rescued": 833, + "harmed": 266, + "missed_rescue": 1131, + "avoided_harm": 905, + "selected_both_correct": 904, + "selected_both_wrong": 700 + }, + { + "budget": 0.2, + "threshold": -2.928896903991699, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.7914119052638977, + "rescued": 870, + "harmed": 285, + "missed_rescue": 1094, + "avoided_harm": 886, + "selected_both_correct": 974, + "selected_both_wrong": 716 + }, + { + "budget": 0.21, + "threshold": -3.0729546546936035, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.7926066483941246, + "rescued": 905, + "harmed": 303, + "missed_rescue": 1059, + "avoided_harm": 868, + "selected_both_correct": 1051, + "selected_both_wrong": 729 + }, + { + "budget": 0.22, + "threshold": -3.207036018371582, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.792817485417106, + "rescued": 925, + "harmed": 320, + "missed_rescue": 1039, + "avoided_harm": 851, + "selected_both_correct": 1140, + "selected_both_wrong": 745 + }, + { + "budget": 0.23, + "threshold": -3.3137195110321045, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.794363623585635, + "rescued": 957, + "harmed": 330, + "missed_rescue": 1007, + "avoided_harm": 841, + "selected_both_correct": 1218, + "selected_both_wrong": 767 + }, + { + "budget": 0.24, + "threshold": -3.4117960929870605, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.7959800407618245, + "rescued": 989, + "harmed": 339, + "missed_rescue": 975, + "avoided_harm": 832, + "selected_both_correct": 1299, + "selected_both_wrong": 787 + }, + { + "budget": 0.25, + "threshold": -3.505338191986084, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.7963314358001266, + "rescued": 1018, + "harmed": 363, + "missed_rescue": 946, + "avoided_harm": 808, + "selected_both_correct": 1374, + "selected_both_wrong": 802 + }, + { + "budget": 0.26, + "threshold": -3.594294548034668, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.7976667369456744, + "rescued": 1044, + "harmed": 370, + "missed_rescue": 920, + "avoided_harm": 801, + "selected_both_correct": 1463, + "selected_both_wrong": 822 + }, + { + "budget": 0.27, + "threshold": -3.6820778846740723, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.7978072949609951, + "rescued": 1066, + "harmed": 390, + "missed_rescue": 898, + "avoided_harm": 781, + "selected_both_correct": 1552, + "selected_both_wrong": 833 + }, + { + "budget": 0.28, + "threshold": -3.7688918113708496, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.7990723170988826, + "rescued": 1096, + "harmed": 402, + "missed_rescue": 868, + "avoided_harm": 769, + "selected_both_correct": 1637, + "selected_both_wrong": 849 + }, + { + "budget": 0.29, + "threshold": -3.8456528186798096, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.7996345491601659, + "rescued": 1115, + "harmed": 413, + "missed_rescue": 849, + "avoided_harm": 758, + "selected_both_correct": 1736, + "selected_both_wrong": 862 + }, + { + "budget": 0.3, + "threshold": -3.9210715293884277, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.799845386183147, + "rescued": 1134, + "harmed": 429, + "missed_rescue": 830, + "avoided_harm": 742, + "selected_both_correct": 1830, + "selected_both_wrong": 875 + }, + { + "budget": 0.31, + "threshold": -4.0025529861450195, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.7997751071754867, + "rescued": 1147, + "harmed": 443, + "missed_rescue": 817, + "avoided_harm": 728, + "selected_both_correct": 1933, + "selected_both_wrong": 887 + }, + { + "budget": 0.32, + "threshold": -4.074873924255371, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.8008995712980532, + "rescued": 1169, + "harmed": 449, + "missed_rescue": 795, + "avoided_harm": 722, + "selected_both_correct": 2035, + "selected_both_wrong": 900 + }, + { + "budget": 0.33, + "threshold": -4.144405841827393, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.8016726403823178, + "rescued": 1192, + "harmed": 461, + "missed_rescue": 772, + "avoided_harm": 710, + "selected_both_correct": 2134, + "selected_both_wrong": 908 + }, + { + "budget": 0.34, + "threshold": -4.213827610015869, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.8027971045048844, + "rescued": 1215, + "harmed": 468, + "missed_rescue": 749, + "avoided_harm": 703, + "selected_both_correct": 2231, + "selected_both_wrong": 923 + }, + { + "budget": 0.35, + "threshold": -4.29035758972168, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.8037810106121301, + "rescued": 1235, + "harmed": 474, + "missed_rescue": 729, + "avoided_harm": 697, + "selected_both_correct": 2335, + "selected_both_wrong": 936 + }, + { + "budget": 0.36, + "threshold": -4.3551530838012695, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.8041324056504322, + "rescued": 1253, + "harmed": 487, + "missed_rescue": 711, + "avoided_harm": 684, + "selected_both_correct": 2435, + "selected_both_wrong": 947 + }, + { + "budget": 0.37, + "threshold": -4.414309978485107, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.8043432426734135, + "rescued": 1272, + "harmed": 503, + "missed_rescue": 692, + "avoided_harm": 668, + "selected_both_correct": 2530, + "selected_both_wrong": 959 + }, + { + "budget": 0.38, + "threshold": -4.483922481536865, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.8037810106121301, + "rescued": 1285, + "harmed": 524, + "missed_rescue": 679, + "avoided_harm": 647, + "selected_both_correct": 2631, + "selected_both_wrong": 967 + }, + { + "budget": 0.39, + "threshold": -4.539813041687012, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.8041324056504322, + "rescued": 1300, + "harmed": 534, + "missed_rescue": 664, + "avoided_harm": 637, + "selected_both_correct": 2738, + "selected_both_wrong": 977 + }, + { + "budget": 0.4, + "threshold": -4.609446048736572, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.8051865907653384, + "rescued": 1323, + "harmed": 542, + "missed_rescue": 641, + "avoided_harm": 629, + "selected_both_correct": 2839, + "selected_both_wrong": 987 + }, + { + "budget": 0.41, + "threshold": -4.663720607757568, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.8053271487806593, + "rescued": 1336, + "harmed": 553, + "missed_rescue": 628, + "avoided_harm": 618, + "selected_both_correct": 2941, + "selected_both_wrong": 1003 + }, + { + "budget": 0.42, + "threshold": -4.7261199951171875, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.8050460327500175, + "rescued": 1348, + "harmed": 569, + "missed_rescue": 616, + "avoided_harm": 602, + "selected_both_correct": 3045, + "selected_both_wrong": 1014 + }, + { + "budget": 0.43, + "threshold": -4.786887168884277, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.805116311757678, + "rescued": 1357, + "harmed": 577, + "missed_rescue": 607, + "avoided_harm": 594, + "selected_both_correct": 3159, + "selected_both_wrong": 1025 + }, + { + "budget": 0.44, + "threshold": -4.843598365783691, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.8055379858036404, + "rescued": 1371, + "harmed": 585, + "missed_rescue": 593, + "avoided_harm": 586, + "selected_both_correct": 3266, + "selected_both_wrong": 1038 + }, + { + "budget": 0.45, + "threshold": -4.890982151031494, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.8057488228266216, + "rescued": 1383, + "harmed": 594, + "missed_rescue": 581, + "avoided_harm": 577, + "selected_both_correct": 3378, + "selected_both_wrong": 1048 + }, + { + "budget": 0.46, + "threshold": -4.9397172927856445, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.8056082648113009, + "rescued": 1392, + "harmed": 605, + "missed_rescue": 572, + "avoided_harm": 566, + "selected_both_correct": 3488, + "selected_both_wrong": 1060 + }, + { + "budget": 0.47, + "threshold": -4.991722583770752, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.8059596598496029, + "rescued": 1407, + "harmed": 615, + "missed_rescue": 557, + "avoided_harm": 556, + "selected_both_correct": 3593, + "selected_both_wrong": 1072 + }, + { + "budget": 0.48, + "threshold": -5.048281669616699, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.8059596598496029, + "rescued": 1420, + "harmed": 628, + "missed_rescue": 544, + "avoided_harm": 543, + "selected_both_correct": 3697, + "selected_both_wrong": 1084 + }, + { + "budget": 0.49, + "threshold": -5.091976165771484, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.8063813338955654, + "rescued": 1435, + "harmed": 637, + "missed_rescue": 529, + "avoided_harm": 534, + "selected_both_correct": 3803, + "selected_both_wrong": 1097 + }, + { + "budget": 0.5, + "threshold": -5.137824535369873, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.8070841239721696, + "rescued": 1449, + "harmed": 641, + "missed_rescue": 515, + "avoided_harm": 530, + "selected_both_correct": 3913, + "selected_both_wrong": 1111 + }, + { + "budget": 0.51, + "threshold": -5.183772563934326, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.8065921709185466, + "rescued": 1454, + "harmed": 653, + "missed_rescue": 510, + "avoided_harm": 518, + "selected_both_correct": 4024, + "selected_both_wrong": 1125 + }, + { + "budget": 0.52, + "threshold": -5.225483417510986, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.8068732869491883, + "rescued": 1465, + "harmed": 660, + "missed_rescue": 499, + "avoided_harm": 511, + "selected_both_correct": 4140, + "selected_both_wrong": 1134 + }, + { + "budget": 0.53, + "threshold": -5.285849571228027, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.8071544029798299, + "rescued": 1479, + "harmed": 670, + "missed_rescue": 485, + "avoided_harm": 501, + "selected_both_correct": 4252, + "selected_both_wrong": 1140 + }, + { + "budget": 0.54, + "threshold": -5.33848237991333, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.807857193056434, + "rescued": 1493, + "harmed": 674, + "missed_rescue": 471, + "avoided_harm": 497, + "selected_both_correct": 4367, + "selected_both_wrong": 1149 + }, + { + "budget": 0.55, + "threshold": -5.383656978607178, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.8079977510717549, + "rescued": 1506, + "harmed": 685, + "missed_rescue": 458, + "avoided_harm": 486, + "selected_both_correct": 4474, + "selected_both_wrong": 1160 + }, + { + "budget": 0.56, + "threshold": -5.435944080352783, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.8081383090870757, + "rescued": 1513, + "harmed": 690, + "missed_rescue": 451, + "avoided_harm": 481, + "selected_both_correct": 4599, + "selected_both_wrong": 1166 + }, + { + "budget": 0.57, + "threshold": -5.478402614593506, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.8079274720640944, + "rescued": 1523, + "harmed": 703, + "missed_rescue": 441, + "avoided_harm": 468, + "selected_both_correct": 4705, + "selected_both_wrong": 1179 + }, + { + "budget": 0.58, + "threshold": -5.5247578620910645, + "operator": "gt", + "actual_fraction": 0.5798720922060581, + "metric": 0.8079274720640944, + "rescued": 1532, + "harmed": 712, + "missed_rescue": 432, + "avoided_harm": 459, + "selected_both_correct": 4819, + "selected_both_wrong": 1188 + }, + { + "budget": 0.59, + "threshold": -5.566192626953125, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.8080680300794153, + "rescued": 1543, + "harmed": 721, + "missed_rescue": 421, + "avoided_harm": 450, + "selected_both_correct": 4933, + "selected_both_wrong": 1198 + }, + { + "budget": 0.6, + "threshold": -5.613105773925781, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.807857193056434, + "rescued": 1551, + "harmed": 732, + "missed_rescue": 413, + "avoided_harm": 439, + "selected_both_correct": 5051, + "selected_both_wrong": 1203 + }, + { + "budget": 0.61, + "threshold": -5.660335540771484, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.8075057980181319, + "rescued": 1557, + "harmed": 743, + "missed_rescue": 407, + "avoided_harm": 428, + "selected_both_correct": 5166, + "selected_both_wrong": 1213 + }, + { + "budget": 0.62, + "threshold": -5.713016986846924, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.8070841239721696, + "rescued": 1560, + "harmed": 752, + "missed_rescue": 404, + "avoided_harm": 419, + "selected_both_correct": 5282, + "selected_both_wrong": 1227 + }, + { + "budget": 0.63, + "threshold": -5.765469074249268, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.8065921709185466, + "rescued": 1568, + "harmed": 767, + "missed_rescue": 396, + "avoided_harm": 404, + "selected_both_correct": 5389, + "selected_both_wrong": 1240 + }, + { + "budget": 0.64, + "threshold": -5.808561325073242, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.8065218919108862, + "rescued": 1579, + "harmed": 779, + "missed_rescue": 385, + "avoided_harm": 392, + "selected_both_correct": 5501, + "selected_both_wrong": 1247 + }, + { + "budget": 0.65, + "threshold": -5.852151870727539, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.8058893808419425, + "rescued": 1584, + "harmed": 793, + "missed_rescue": 380, + "avoided_harm": 378, + "selected_both_correct": 5618, + "selected_both_wrong": 1253 + }, + { + "budget": 0.66, + "threshold": -5.899008274078369, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.8056785438189613, + "rescued": 1593, + "harmed": 805, + "missed_rescue": 371, + "avoided_harm": 366, + "selected_both_correct": 5729, + "selected_both_wrong": 1264 + }, + { + "budget": 0.67, + "threshold": -5.944814682006836, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.8057488228266216, + "rescued": 1603, + "harmed": 814, + "missed_rescue": 361, + "avoided_harm": 357, + "selected_both_correct": 5849, + "selected_both_wrong": 1267 + }, + { + "budget": 0.68, + "threshold": -5.992681980133057, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.8060299388572634, + "rescued": 1620, + "harmed": 827, + "missed_rescue": 344, + "avoided_harm": 344, + "selected_both_correct": 5954, + "selected_both_wrong": 1274 + }, + { + "budget": 0.69, + "threshold": -6.03839635848999, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.8056082648113009, + "rescued": 1629, + "harmed": 842, + "missed_rescue": 335, + "avoided_harm": 329, + "selected_both_correct": 6066, + "selected_both_wrong": 1281 + }, + { + "budget": 0.7, + "threshold": -6.091725826263428, + "operator": "ge", + "actual_fraction": 0.6999789162977019, + "metric": 0.8055379858036404, + "rescued": 1635, + "harmed": 849, + "missed_rescue": 329, + "avoided_harm": 322, + "selected_both_correct": 6187, + "selected_both_wrong": 1289 + }, + { + "budget": 0.71, + "threshold": -6.139003276824951, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.8053974277883196, + "rescued": 1644, + "harmed": 860, + "missed_rescue": 320, + "avoided_harm": 311, + "selected_both_correct": 6302, + "selected_both_wrong": 1296 + }, + { + "budget": 0.72, + "threshold": -6.183269500732422, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.80546770679598, + "rescued": 1655, + "harmed": 870, + "missed_rescue": 309, + "avoided_harm": 301, + "selected_both_correct": 6417, + "selected_both_wrong": 1302 + }, + { + "budget": 0.73, + "threshold": -6.240170955657959, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.8055379858036404, + "rescued": 1664, + "harmed": 878, + "missed_rescue": 300, + "avoided_harm": 293, + "selected_both_correct": 6531, + "selected_both_wrong": 1314 + }, + { + "budget": 0.74, + "threshold": -6.2856950759887695, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.8058191018342821, + "rescued": 1676, + "harmed": 886, + "missed_rescue": 288, + "avoided_harm": 285, + "selected_both_correct": 6650, + "selected_both_wrong": 1317 + }, + { + "budget": 0.75, + "threshold": -6.338753700256348, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.8057488228266216, + "rescued": 1684, + "harmed": 895, + "missed_rescue": 280, + "avoided_harm": 276, + "selected_both_correct": 6764, + "selected_both_wrong": 1328 + }, + { + "budget": 0.76, + "threshold": -6.385316848754883, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.8058893808419425, + "rescued": 1695, + "harmed": 904, + "missed_rescue": 269, + "avoided_harm": 267, + "selected_both_correct": 6872, + "selected_both_wrong": 1343 + }, + { + "budget": 0.77, + "threshold": -6.438819408416748, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.8061704968725841, + "rescued": 1707, + "harmed": 912, + "missed_rescue": 257, + "avoided_harm": 259, + "selected_both_correct": 6987, + "selected_both_wrong": 1350 + }, + { + "budget": 0.78, + "threshold": -6.489089012145996, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.8062407758802446, + "rescued": 1719, + "harmed": 923, + "missed_rescue": 245, + "avoided_harm": 248, + "selected_both_correct": 7096, + "selected_both_wrong": 1360 + }, + { + "budget": 0.79, + "threshold": -6.554056644439697, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.8066624499262071, + "rescued": 1729, + "harmed": 927, + "missed_rescue": 235, + "avoided_harm": 244, + "selected_both_correct": 7218, + "selected_both_wrong": 1366 + }, + { + "budget": 0.8, + "threshold": -6.614213943481445, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.8068030079415278, + "rescued": 1740, + "harmed": 936, + "missed_rescue": 224, + "avoided_harm": 235, + "selected_both_correct": 7333, + "selected_both_wrong": 1374 + }, + { + "budget": 0.81, + "threshold": -6.680136680603027, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.8072949609951507, + "rescued": 1756, + "harmed": 945, + "missed_rescue": 208, + "avoided_harm": 226, + "selected_both_correct": 7439, + "selected_both_wrong": 1385 + }, + { + "budget": 0.82, + "threshold": -6.735454559326172, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.8072246819874903, + "rescued": 1761, + "harmed": 951, + "missed_rescue": 203, + "avoided_harm": 220, + "selected_both_correct": 7561, + "selected_both_wrong": 1394 + }, + { + "budget": 0.83, + "threshold": -6.790655136108398, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.8075760770257924, + "rescued": 1770, + "harmed": 955, + "missed_rescue": 194, + "avoided_harm": 216, + "selected_both_correct": 7684, + "selected_both_wrong": 1401 + }, + { + "budget": 0.84, + "threshold": -6.856695175170898, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.8075760770257924, + "rescued": 1783, + "harmed": 968, + "missed_rescue": 181, + "avoided_harm": 203, + "selected_both_correct": 7793, + "selected_both_wrong": 1408 + }, + { + "budget": 0.85, + "threshold": -6.932329177856445, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.8075760770257924, + "rescued": 1792, + "harmed": 977, + "missed_rescue": 172, + "avoided_harm": 194, + "selected_both_correct": 7906, + "selected_both_wrong": 1419 + }, + { + "budget": 0.86, + "threshold": -6.999267578125, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.8077166350411132, + "rescued": 1799, + "harmed": 982, + "missed_rescue": 165, + "avoided_harm": 189, + "selected_both_correct": 8025, + "selected_both_wrong": 1430 + }, + { + "budget": 0.87, + "threshold": -7.081692695617676, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.8077166350411132, + "rescued": 1807, + "harmed": 990, + "missed_rescue": 157, + "avoided_harm": 181, + "selected_both_correct": 8145, + "selected_both_wrong": 1437 + }, + { + "budget": 0.88, + "threshold": -7.155144214630127, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.8077869140487737, + "rescued": 1819, + "harmed": 1001, + "missed_rescue": 145, + "avoided_harm": 170, + "selected_both_correct": 8259, + "selected_both_wrong": 1442 + }, + { + "budget": 0.89, + "threshold": -7.227406978607178, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.8076463560334528, + "rescued": 1829, + "harmed": 1013, + "missed_rescue": 135, + "avoided_harm": 158, + "selected_both_correct": 8365, + "selected_both_wrong": 1456 + }, + { + "budget": 0.9, + "threshold": -7.308749198913574, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.8070138449645091, + "rescued": 1837, + "harmed": 1030, + "missed_rescue": 127, + "avoided_harm": 141, + "selected_both_correct": 8472, + "selected_both_wrong": 1467 + }, + { + "budget": 0.91, + "threshold": -7.405246257781982, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.8070138449645091, + "rescued": 1848, + "harmed": 1041, + "missed_rescue": 116, + "avoided_harm": 130, + "selected_both_correct": 8582, + "selected_both_wrong": 1477 + }, + { + "budget": 0.92, + "threshold": -7.509663105010986, + "operator": "gt", + "actual_fraction": 0.9198819312671305, + "metric": 0.8070138449645091, + "rescued": 1856, + "harmed": 1049, + "missed_rescue": 108, + "avoided_harm": 122, + "selected_both_correct": 8695, + "selected_both_wrong": 1489 + }, + { + "budget": 0.93, + "threshold": -7.625953674316406, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.8068030079415278, + "rescued": 1865, + "harmed": 1061, + "missed_rescue": 99, + "avoided_harm": 110, + "selected_both_correct": 8805, + "selected_both_wrong": 1501 + }, + { + "budget": 0.94, + "threshold": -7.762190818786621, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.8068030079415278, + "rescued": 1878, + "harmed": 1074, + "missed_rescue": 86, + "avoided_harm": 97, + "selected_both_correct": 8911, + "selected_both_wrong": 1512 + }, + { + "budget": 0.95, + "threshold": -7.891505241394043, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.8068732869491883, + "rescued": 1894, + "harmed": 1089, + "missed_rescue": 70, + "avoided_harm": 82, + "selected_both_correct": 9012, + "selected_both_wrong": 1522 + }, + { + "budget": 0.96, + "threshold": -8.049057960510254, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.8068732869491883, + "rescued": 1913, + "harmed": 1108, + "missed_rescue": 51, + "avoided_harm": 63, + "selected_both_correct": 9108, + "selected_both_wrong": 1530 + }, + { + "budget": 0.97, + "threshold": -8.248269081115723, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.8071544029798299, + "rescued": 1927, + "harmed": 1118, + "missed_rescue": 37, + "avoided_harm": 53, + "selected_both_correct": 9215, + "selected_both_wrong": 1542 + }, + { + "budget": 0.98, + "threshold": -8.543853759765625, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.8068030079415278, + "rescued": 1939, + "harmed": 1135, + "missed_rescue": 25, + "avoided_harm": 36, + "selected_both_correct": 9310, + "selected_both_wrong": 1560 + }, + { + "budget": 0.99, + "threshold": -8.96818733215332, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.8065218919108862, + "rescued": 1948, + "harmed": 1148, + "missed_rescue": 16, + "avoided_harm": 23, + "selected_both_correct": 9420, + "selected_both_wrong": 1570 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8060299388572634, + "rescued": 1964, + "harmed": 1171, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9505, + "selected_both_wrong": 1589 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.6695119530385957, + "rescued": 0, + "harmed": 0, + "missed_rescue": 5423, + "avoided_harm": 2771, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 5.873929023742676, + "operator": "ge", + "actual_fraction": 0.04701812097666109, + "metric": 0.6878314380512152, + "rescued": 655, + "harmed": 9, + "missed_rescue": 4768, + "avoided_harm": 2762, + "selected_both_correct": 18, + "selected_both_wrong": 976 + }, + { + "budget": 0.02, + "threshold": 3.9572336673736572, + "operator": "ge", + "actual_fraction": 0.06706746448118424, + "metric": 0.6980404389870403, + "rescued": 1022, + "harmed": 16, + "missed_rescue": 4401, + "avoided_harm": 2755, + "selected_both_correct": 64, + "selected_both_wrong": 1263 + }, + { + "budget": 0.03, + "threshold": 2.499727725982666, + "operator": "ge", + "actual_fraction": 0.08720188299350594, + "metric": 0.7065195814309616, + "rescued": 1340, + "harmed": 35, + "missed_rescue": 4083, + "avoided_harm": 2736, + "selected_both_correct": 157, + "selected_both_wrong": 1543 + }, + { + "budget": 0.04, + "threshold": 1.2223308086395264, + "operator": "ge", + "actual_fraction": 0.1055780846779911, + "metric": 0.712389756969061, + "rescued": 1582, + "harmed": 70, + "missed_rescue": 3841, + "avoided_harm": 2701, + "selected_both_correct": 311, + "selected_both_wrong": 1760 + }, + { + "budget": 0.05, + "threshold": 0.36731216311454773, + "operator": "ge", + "actual_fraction": 0.11998411933187761, + "metric": 0.7164449990074583, + "rescued": 1757, + "harmed": 102, + "missed_rescue": 3666, + "avoided_harm": 2669, + "selected_both_correct": 475, + "selected_both_wrong": 1897 + }, + { + "budget": 0.06, + "threshold": -0.14144867658615112, + "operator": "ge", + "actual_fraction": 0.13243342880639764, + "metric": 0.7199047159912656, + "rescued": 1911, + "harmed": 134, + "missed_rescue": 3512, + "avoided_harm": 2637, + "selected_both_correct": 596, + "selected_both_wrong": 2029 + }, + { + "budget": 0.07, + "threshold": -0.42117512226104736, + "operator": "ge", + "actual_fraction": 0.14235884638289426, + "metric": 0.7222017412018262, + "rescued": 2025, + "harmed": 167, + "missed_rescue": 3398, + "avoided_harm": 2604, + "selected_both_correct": 705, + "selected_both_wrong": 2123 + }, + { + "budget": 0.08, + "threshold": -0.6775990128517151, + "operator": "ge", + "actual_fraction": 0.1511215721861441, + "metric": 0.7239032413577972, + "rescued": 2118, + "harmed": 200, + "missed_rescue": 3305, + "avoided_harm": 2571, + "selected_both_correct": 789, + "selected_both_wrong": 2222 + }, + { + "budget": 0.09, + "threshold": -0.9111056327819824, + "operator": "ge", + "actual_fraction": 0.15957235629413266, + "metric": 0.7263136999120892, + "rescued": 2226, + "harmed": 223, + "missed_rescue": 3197, + "avoided_harm": 2548, + "selected_both_correct": 857, + "selected_both_wrong": 2321 + }, + { + "budget": 0.1, + "threshold": -1.1332147121429443, + "operator": "ge", + "actual_fraction": 0.16822164875365114, + "metric": 0.7290077418257097, + "rescued": 2339, + "harmed": 241, + "missed_rescue": 3084, + "avoided_harm": 2530, + "selected_both_correct": 914, + "selected_both_wrong": 2438 + }, + { + "budget": 0.11, + "threshold": -1.3457791805267334, + "operator": "ge", + "actual_fraction": 0.17695601622096815, + "metric": 0.7316450670674645, + "rescued": 2454, + "harmed": 263, + "missed_rescue": 2969, + "avoided_harm": 2508, + "selected_both_correct": 997, + "selected_both_wrong": 2526 + }, + { + "budget": 0.12, + "threshold": -1.5916669368743896, + "operator": "ge", + "actual_fraction": 0.18855457561750277, + "metric": 0.7345943340044806, + "rescued": 2587, + "harmed": 292, + "missed_rescue": 2836, + "avoided_harm": 2479, + "selected_both_correct": 1103, + "selected_both_wrong": 2667 + }, + { + "budget": 0.13, + "threshold": -1.7836966514587402, + "operator": "ge", + "actual_fraction": 0.19751580977228256, + "metric": 0.7366361341916456, + "rescued": 2685, + "harmed": 318, + "missed_rescue": 2738, + "avoided_harm": 2453, + "selected_both_correct": 1187, + "selected_both_wrong": 2775 + }, + { + "budget": 0.14, + "threshold": -1.9795069694519043, + "operator": "ge", + "actual_fraction": 0.20664719394265946, + "metric": 0.7385361426991465, + "rescued": 2776, + "harmed": 342, + "missed_rescue": 2647, + "avoided_harm": 2429, + "selected_both_correct": 1271, + "selected_both_wrong": 2898 + }, + { + "budget": 0.15, + "threshold": -2.1732606887817383, + "operator": "ge", + "actual_fraction": 0.2172532115815444, + "metric": 0.7402376428551173, + "rescued": 2874, + "harmed": 380, + "missed_rescue": 2549, + "avoided_harm": 2391, + "selected_both_correct": 1386, + "selected_both_wrong": 3021 + }, + { + "budget": 0.16, + "threshold": -2.3698818683624268, + "operator": "gt", + "actual_fraction": 0.22717862915804102, + "metric": 0.7420242180188866, + "rescued": 2970, + "harmed": 413, + "missed_rescue": 2453, + "avoided_harm": 2358, + "selected_both_correct": 1489, + "selected_both_wrong": 3139 + }, + { + "budget": 0.17, + "threshold": -2.5250070095062256, + "operator": "ge", + "actual_fraction": 0.2361398633128208, + "metric": 0.7436122848311261, + "rescued": 3053, + "harmed": 440, + "missed_rescue": 2370, + "avoided_harm": 2331, + "selected_both_correct": 1581, + "selected_both_wrong": 3253 + }, + { + "budget": 0.18, + "threshold": -2.659658908843994, + "operator": "ge", + "actual_fraction": 0.24362646399909255, + "metric": 0.7448033349403057, + "rescued": 3124, + "harmed": 469, + "missed_rescue": 2299, + "avoided_harm": 2302, + "selected_both_correct": 1685, + "selected_both_wrong": 3313 + }, + { + "budget": 0.19, + "threshold": -2.7833452224731445, + "operator": "ge", + "actual_fraction": 0.2507727646541701, + "metric": 0.7456540850182911, + "rescued": 3179, + "harmed": 494, + "missed_rescue": 2244, + "avoided_harm": 2277, + "selected_both_correct": 1781, + "selected_both_wrong": 3389 + }, + { + "budget": 0.2, + "threshold": -2.928896903991699, + "operator": "ge", + "actual_fraction": 0.25942205711368854, + "metric": 0.7459943850494853, + "rescued": 3237, + "harmed": 540, + "missed_rescue": 2186, + "avoided_harm": 2231, + "selected_both_correct": 1913, + "selected_both_wrong": 3458 + }, + { + "budget": 0.21, + "threshold": -3.0729546546936035, + "operator": "ge", + "actual_fraction": 0.2690922496667896, + "metric": 0.746419760088478, + "rescued": 3296, + "harmed": 584, + "missed_rescue": 2127, + "avoided_harm": 2187, + "selected_both_correct": 2081, + "selected_both_wrong": 3528 + }, + { + "budget": 0.22, + "threshold": -3.207036018371582, + "operator": "ge", + "actual_fraction": 0.2781385588293679, + "metric": 0.7467600601196722, + "rescued": 3347, + "harmed": 623, + "missed_rescue": 2076, + "avoided_harm": 2148, + "selected_both_correct": 2230, + "selected_both_wrong": 3608 + }, + { + "budget": 0.23, + "threshold": -3.3137195110321045, + "operator": "ge", + "actual_fraction": 0.2868445679607521, + "metric": 0.7473555851742619, + "rescued": 3396, + "harmed": 651, + "missed_rescue": 2027, + "avoided_harm": 2120, + "selected_both_correct": 2391, + "selected_both_wrong": 3677 + }, + { + "budget": 0.24, + "threshold": -3.4117960929870605, + "operator": "ge", + "actual_fraction": 0.29430281031109096, + "metric": 0.7480645435725831, + "rescued": 3446, + "harmed": 676, + "missed_rescue": 1977, + "avoided_harm": 2095, + "selected_both_correct": 2535, + "selected_both_wrong": 3721 + }, + { + "budget": 0.25, + "threshold": -3.505338191986084, + "operator": "ge", + "actual_fraction": 0.30176105266142983, + "metric": 0.748092901908516, + "rescued": 3479, + "harmed": 708, + "missed_rescue": 1944, + "avoided_harm": 2063, + "selected_both_correct": 2673, + "selected_both_wrong": 3781 + }, + { + "budget": 0.26, + "threshold": -3.594294548034668, + "operator": "ge", + "actual_fraction": 0.30995661174602274, + "metric": 0.7486033519553073, + "rescued": 3520, + "harmed": 731, + "missed_rescue": 1903, + "avoided_harm": 2040, + "selected_both_correct": 2847, + "selected_both_wrong": 3832 + }, + { + "budget": 0.27, + "threshold": -3.6820778846740723, + "operator": "ge", + "actual_fraction": 0.3175850041119587, + "metric": 0.7494541020332927, + "rescued": 3569, + "harmed": 750, + "missed_rescue": 1854, + "avoided_harm": 2021, + "selected_both_correct": 3000, + "selected_both_wrong": 3880 + }, + { + "budget": 0.28, + "threshold": -3.7688918113708496, + "operator": "ge", + "actual_fraction": 0.3262342965714772, + "metric": 0.7501630604316138, + "rescued": 3623, + "harmed": 779, + "missed_rescue": 1800, + "avoided_harm": 1992, + "selected_both_correct": 3177, + "selected_both_wrong": 3925 + }, + { + "budget": 0.29, + "threshold": -3.8456528186798096, + "operator": "ge", + "actual_fraction": 0.3324731304767036, + "metric": 0.7505600771346738, + "rescued": 3655, + "harmed": 797, + "missed_rescue": 1768, + "avoided_harm": 1974, + "selected_both_correct": 3301, + "selected_both_wrong": 3971 + }, + { + "budget": 0.3, + "threshold": -3.9210715293884277, + "operator": "ge", + "actual_fraction": 0.3399313728270425, + "metric": 0.7509570938377336, + "rescued": 3698, + "harmed": 826, + "missed_rescue": 1725, + "avoided_harm": 1945, + "selected_both_correct": 3454, + "selected_both_wrong": 4009 + }, + { + "budget": 0.31, + "threshold": -4.0025529861450195, + "operator": "ge", + "actual_fraction": 0.34824036525536683, + "metric": 0.7511839605251964, + "rescued": 3732, + "harmed": 852, + "missed_rescue": 1691, + "avoided_harm": 1919, + "selected_both_correct": 3635, + "selected_both_wrong": 4061 + }, + { + "budget": 0.32, + "threshold": -4.074873924255371, + "operator": "ge", + "actual_fraction": 0.3557553242775714, + "metric": 0.751269035532995, + "rescued": 3761, + "harmed": 878, + "missed_rescue": 1662, + "avoided_harm": 1893, + "selected_both_correct": 3798, + "selected_both_wrong": 4108 + }, + { + "budget": 0.33, + "threshold": -4.144405841827393, + "operator": "ge", + "actual_fraction": 0.3627598332529847, + "metric": 0.7511272438533307, + "rescued": 3779, + "harmed": 901, + "missed_rescue": 1644, + "avoided_harm": 1870, + "selected_both_correct": 3962, + "selected_both_wrong": 4150 + }, + { + "budget": 0.34, + "threshold": -4.213827610015869, + "operator": "ge", + "actual_fraction": 0.37013300059552506, + "metric": 0.7512123188611293, + "rescued": 3806, + "harmed": 925, + "missed_rescue": 1617, + "avoided_harm": 1846, + "selected_both_correct": 4131, + "selected_both_wrong": 4190 + }, + { + "budget": 0.35, + "threshold": -4.29035758972168, + "operator": "ge", + "actual_fraction": 0.3783569180160508, + "metric": 0.751439185548592, + "rescued": 3839, + "harmed": 950, + "missed_rescue": 1584, + "avoided_harm": 1821, + "selected_both_correct": 4321, + "selected_both_wrong": 4232 + }, + { + "budget": 0.36, + "threshold": -4.3551530838012695, + "operator": "ge", + "actual_fraction": 0.3854748603351955, + "metric": 0.7517511272438533, + "rescued": 3869, + "harmed": 969, + "missed_rescue": 1554, + "avoided_harm": 1802, + "selected_both_correct": 4491, + "selected_both_wrong": 4264 + }, + { + "budget": 0.37, + "threshold": -4.414309978485107, + "operator": "ge", + "actual_fraction": 0.39279131100587017, + "metric": 0.752176502282846, + "rescued": 3898, + "harmed": 983, + "missed_rescue": 1525, + "avoided_harm": 1788, + "selected_both_correct": 4671, + "selected_both_wrong": 4299 + }, + { + "budget": 0.38, + "threshold": -4.483922481536865, + "operator": "ge", + "actual_fraction": 0.40013612001247767, + "metric": 0.7522332189547117, + "rescued": 3921, + "harmed": 1004, + "missed_rescue": 1502, + "avoided_harm": 1767, + "selected_both_correct": 4853, + "selected_both_wrong": 4332 + }, + { + "budget": 0.39, + "threshold": -4.539813041687012, + "operator": "ge", + "actual_fraction": 0.406573462269234, + "metric": 0.7525735189859059, + "rescued": 3947, + "harmed": 1018, + "missed_rescue": 1476, + "avoided_harm": 1753, + "selected_both_correct": 5011, + "selected_both_wrong": 4361 + }, + { + "budget": 0.4, + "threshold": -4.609446048736572, + "operator": "ge", + "actual_fraction": 0.4153929047443496, + "metric": 0.7528003856733687, + "rescued": 3980, + "harmed": 1043, + "missed_rescue": 1443, + "avoided_harm": 1728, + "selected_both_correct": 5214, + "selected_both_wrong": 4411 + }, + { + "budget": 0.41, + "threshold": -4.663720607757568, + "operator": "ge", + "actual_fraction": 0.42151830530584466, + "metric": 0.75311232736863, + "rescued": 4010, + "harmed": 1062, + "missed_rescue": 1413, + "avoided_harm": 1709, + "selected_both_correct": 5358, + "selected_both_wrong": 4434 + }, + { + "budget": 0.42, + "threshold": -4.7261199951171875, + "operator": "ge", + "actual_fraction": 0.42920341434364634, + "metric": 0.7535093440716899, + "rescued": 4045, + "harmed": 1083, + "missed_rescue": 1378, + "avoided_harm": 1688, + "selected_both_correct": 5538, + "selected_both_wrong": 4469 + }, + { + "budget": 0.43, + "threshold": -4.786887168884277, + "operator": "ge", + "actual_fraction": 0.4367750900377166, + "metric": 0.7537362107591526, + "rescued": 4070, + "harmed": 1100, + "missed_rescue": 1353, + "avoided_harm": 1671, + "selected_both_correct": 5721, + "selected_both_wrong": 4511 + }, + { + "budget": 0.44, + "threshold": -4.843598365783691, + "operator": "ge", + "actual_fraction": 0.4450273657941752, + "metric": 0.753679494087287, + "rescued": 4099, + "harmed": 1131, + "missed_rescue": 1324, + "avoided_harm": 1640, + "selected_both_correct": 5910, + "selected_both_wrong": 4553 + }, + { + "budget": 0.45, + "threshold": -4.890982151031494, + "operator": "ge", + "actual_fraction": 0.45143634971499874, + "metric": 0.7537929274310183, + "rescued": 4122, + "harmed": 1150, + "missed_rescue": 1301, + "avoided_harm": 1621, + "selected_both_correct": 6067, + "selected_both_wrong": 4580 + }, + { + "budget": 0.46, + "threshold": -4.9397172927856445, + "operator": "ge", + "actual_fraction": 0.4578736919717551, + "metric": 0.7536227774154213, + "rescued": 4144, + "harmed": 1178, + "missed_rescue": 1279, + "avoided_harm": 1593, + "selected_both_correct": 6218, + "selected_both_wrong": 4606 + }, + { + "budget": 0.47, + "threshold": -4.991722583770752, + "operator": "ge", + "actual_fraction": 0.4659274593766838, + "metric": 0.7537929274310183, + "rescued": 4172, + "harmed": 1200, + "missed_rescue": 1251, + "avoided_harm": 1571, + "selected_both_correct": 6425, + "selected_both_wrong": 4633 + }, + { + "budget": 0.48, + "threshold": -5.048281669616699, + "operator": "ge", + "actual_fraction": 0.4741797351331424, + "metric": 0.7538780024388169, + "rescued": 4202, + "harmed": 1227, + "missed_rescue": 1221, + "avoided_harm": 1544, + "selected_both_correct": 6628, + "selected_both_wrong": 4664 + }, + { + "budget": 0.49, + "threshold": -5.091976165771484, + "operator": "ge", + "actual_fraction": 0.4806454357258316, + "metric": 0.7537362107591526, + "rescued": 4222, + "harmed": 1252, + "missed_rescue": 1201, + "avoided_harm": 1519, + "selected_both_correct": 6781, + "selected_both_wrong": 4694 + }, + { + "budget": 0.5, + "threshold": -5.137824535369873, + "operator": "ge", + "actual_fraction": 0.487451436349715, + "metric": 0.7535093440716899, + "rescued": 4237, + "harmed": 1275, + "missed_rescue": 1186, + "avoided_harm": 1496, + "selected_both_correct": 6963, + "selected_both_wrong": 4714 + }, + { + "budget": 0.51, + "threshold": -5.183772563934326, + "operator": "ge", + "actual_fraction": 0.49482460369225534, + "metric": 0.7537645690950855, + "rescued": 4267, + "harmed": 1296, + "missed_rescue": 1156, + "avoided_harm": 1475, + "selected_both_correct": 7140, + "selected_both_wrong": 4746 + }, + { + "budget": 0.52, + "threshold": -5.225483417510986, + "operator": "ge", + "actual_fraction": 0.501573887644273, + "metric": 0.7535660607435556, + "rescued": 4282, + "harmed": 1318, + "missed_rescue": 1141, + "avoided_harm": 1453, + "selected_both_correct": 7308, + "selected_both_wrong": 4779 + }, + { + "budget": 0.53, + "threshold": -5.285849571228027, + "operator": "ge", + "actual_fraction": 0.5105351217990528, + "metric": 0.7532257607123614, + "rescued": 4298, + "harmed": 1346, + "missed_rescue": 1125, + "avoided_harm": 1425, + "selected_both_correct": 7555, + "selected_both_wrong": 4804 + }, + { + "budget": 0.54, + "threshold": -5.33848237991333, + "operator": "ge", + "actual_fraction": 0.519241130930437, + "metric": 0.7530556106967643, + "rescued": 4319, + "harmed": 1373, + "missed_rescue": 1104, + "avoided_harm": 1398, + "selected_both_correct": 7769, + "selected_both_wrong": 4849 + }, + { + "budget": 0.55, + "threshold": -5.383656978607178, + "operator": "ge", + "actual_fraction": 0.5268128066245072, + "metric": 0.7528571023452344, + "rescued": 4337, + "harmed": 1398, + "missed_rescue": 1086, + "avoided_harm": 1373, + "selected_both_correct": 7963, + "selected_both_wrong": 4879 + }, + { + "budget": 0.56, + "threshold": -5.435944080352783, + "operator": "ge", + "actual_fraction": 0.5355755324277571, + "metric": 0.7528287440093016, + "rescued": 4361, + "harmed": 1423, + "missed_rescue": 1062, + "avoided_harm": 1348, + "selected_both_correct": 8197, + "selected_both_wrong": 4905 + }, + { + "budget": 0.57, + "threshold": -5.478402614593506, + "operator": "ge", + "actual_fraction": 0.5419561580126478, + "metric": 0.753282477384227, + "rescued": 4390, + "harmed": 1436, + "missed_rescue": 1033, + "avoided_harm": 1335, + "selected_both_correct": 8356, + "selected_both_wrong": 4929 + }, + { + "budget": 0.58, + "threshold": -5.5247578620910645, + "operator": "gt", + "actual_fraction": 0.5495845503785838, + "metric": 0.7528287440093016, + "rescued": 4406, + "harmed": 1468, + "missed_rescue": 1017, + "avoided_harm": 1303, + "selected_both_correct": 8545, + "selected_both_wrong": 4961 + }, + { + "budget": 0.59, + "threshold": -5.566192626953125, + "operator": "ge", + "actual_fraction": 0.5562771176587358, + "metric": 0.7527153106655702, + "rescued": 4430, + "harmed": 1496, + "missed_rescue": 993, + "avoided_harm": 1275, + "selected_both_correct": 8706, + "selected_both_wrong": 4984 + }, + { + "budget": 0.6, + "threshold": -5.613105773925781, + "operator": "ge", + "actual_fraction": 0.564047301704336, + "metric": 0.7526585939937045, + "rescued": 4452, + "harmed": 1520, + "missed_rescue": 971, + "avoided_harm": 1251, + "selected_both_correct": 8905, + "selected_both_wrong": 5013 + }, + { + "budget": 0.61, + "threshold": -5.660335540771484, + "operator": "ge", + "actual_fraction": 0.5725831608201231, + "metric": 0.7528287440093016, + "rescued": 4481, + "harmed": 1543, + "missed_rescue": 942, + "avoided_harm": 1228, + "selected_both_correct": 9128, + "selected_both_wrong": 5039 + }, + { + "budget": 0.62, + "threshold": -5.713016986846924, + "operator": "ge", + "actual_fraction": 0.5818846950060971, + "metric": 0.7528571023452344, + "rescued": 4511, + "harmed": 1572, + "missed_rescue": 912, + "avoided_harm": 1199, + "selected_both_correct": 9371, + "selected_both_wrong": 5065 + }, + { + "budget": 0.63, + "threshold": -5.765469074249268, + "operator": "ge", + "actual_fraction": 0.5903638374500184, + "metric": 0.7528571023452344, + "rescued": 4535, + "harmed": 1596, + "missed_rescue": 888, + "avoided_harm": 1175, + "selected_both_correct": 9596, + "selected_both_wrong": 5091 + }, + { + "budget": 0.64, + "threshold": -5.808561325073242, + "operator": "ge", + "actual_fraction": 0.5973683464254318, + "metric": 0.7526869523296373, + "rescued": 4548, + "harmed": 1615, + "missed_rescue": 875, + "avoided_harm": 1156, + "selected_both_correct": 9794, + "selected_both_wrong": 5108 + }, + { + "budget": 0.65, + "threshold": -5.852151870727539, + "operator": "ge", + "actual_fraction": 0.6042594220571137, + "metric": 0.7526018773218388, + "rescued": 4570, + "harmed": 1640, + "missed_rescue": 853, + "avoided_harm": 1131, + "selected_both_correct": 9969, + "selected_both_wrong": 5129 + }, + { + "budget": 0.66, + "threshold": -5.899008274078369, + "operator": "ge", + "actual_fraction": 0.6122848311261095, + "metric": 0.752545160649973, + "rescued": 4589, + "harmed": 1661, + "missed_rescue": 834, + "avoided_harm": 1110, + "selected_both_correct": 10189, + "selected_both_wrong": 5152 + }, + { + "budget": 0.67, + "threshold": -5.944814682006836, + "operator": "ge", + "actual_fraction": 0.6198848651561126, + "metric": 0.7522048606187789, + "rescued": 4603, + "harmed": 1687, + "missed_rescue": 820, + "avoided_harm": 1084, + "selected_both_correct": 10390, + "selected_both_wrong": 5179 + }, + { + "budget": 0.68, + "threshold": -5.992681980133057, + "operator": "ge", + "actual_fraction": 0.6280520659047727, + "metric": 0.752176502282846, + "rescued": 4622, + "harmed": 1707, + "missed_rescue": 801, + "avoided_harm": 1064, + "selected_both_correct": 10612, + "selected_both_wrong": 5206 + }, + { + "budget": 0.69, + "threshold": -6.03839635848999, + "operator": "ge", + "actual_fraction": 0.6363894166690298, + "metric": 0.7522048606187789, + "rescued": 4651, + "harmed": 1735, + "missed_rescue": 772, + "avoided_harm": 1036, + "selected_both_correct": 10825, + "selected_both_wrong": 5230 + }, + { + "budget": 0.7, + "threshold": -6.091725826263428, + "operator": "ge", + "actual_fraction": 0.6444999007458242, + "metric": 0.7521197856109804, + "rescued": 4673, + "harmed": 1760, + "missed_rescue": 750, + "avoided_harm": 1011, + "selected_both_correct": 11035, + "selected_both_wrong": 5259 + }, + { + "budget": 0.71, + "threshold": -6.139003276824951, + "operator": "ge", + "actual_fraction": 0.6527521765022829, + "metric": 0.7519212772594505, + "rescued": 4686, + "harmed": 1780, + "missed_rescue": 737, + "avoided_harm": 991, + "selected_both_correct": 11264, + "selected_both_wrong": 5288 + }, + { + "budget": 0.72, + "threshold": -6.183269500732422, + "operator": "ge", + "actual_fraction": 0.6601253438448231, + "metric": 0.7518362022516518, + "rescued": 4704, + "harmed": 1801, + "missed_rescue": 719, + "avoided_harm": 970, + "selected_both_correct": 11458, + "selected_both_wrong": 5315 + }, + { + "budget": 0.73, + "threshold": -6.240170955657959, + "operator": "ge", + "actual_fraction": 0.6697955363979242, + "metric": 0.7518362022516518, + "rescued": 4732, + "harmed": 1829, + "missed_rescue": 691, + "avoided_harm": 942, + "selected_both_correct": 11714, + "selected_both_wrong": 5344 + }, + { + "budget": 0.74, + "threshold": -6.2856950759887695, + "operator": "ge", + "actual_fraction": 0.6775373621075915, + "metric": 0.7518645605875848, + "rescued": 4754, + "harmed": 1850, + "missed_rescue": 669, + "avoided_harm": 921, + "selected_both_correct": 11918, + "selected_both_wrong": 5370 + }, + { + "budget": 0.75, + "threshold": -6.338753700256348, + "operator": "ge", + "actual_fraction": 0.6870657629810283, + "metric": 0.7512973938689278, + "rescued": 4769, + "harmed": 1885, + "missed_rescue": 654, + "avoided_harm": 886, + "selected_both_correct": 12181, + "selected_both_wrong": 5393 + }, + { + "budget": 0.76, + "threshold": -6.385316848754883, + "operator": "ge", + "actual_fraction": 0.6956299804327482, + "metric": 0.7514108272126592, + "rescued": 4786, + "harmed": 1898, + "missed_rescue": 637, + "avoided_harm": 873, + "selected_both_correct": 12430, + "selected_both_wrong": 5416 + }, + { + "budget": 0.77, + "threshold": -6.438819408416748, + "operator": "ge", + "actual_fraction": 0.7044494229078637, + "metric": 0.7514959022204577, + "rescued": 4805, + "harmed": 1914, + "missed_rescue": 618, + "avoided_harm": 857, + "selected_both_correct": 12668, + "selected_both_wrong": 5454 + }, + { + "budget": 0.78, + "threshold": -6.489089012145996, + "operator": "ge", + "actual_fraction": 0.7129852820236509, + "metric": 0.7510705271814649, + "rescued": 4813, + "harmed": 1937, + "missed_rescue": 610, + "avoided_harm": 834, + "selected_both_correct": 12915, + "selected_both_wrong": 5477 + }, + { + "budget": 0.79, + "threshold": -6.554056644439697, + "operator": "ge", + "actual_fraction": 0.7243853330686555, + "metric": 0.7506735104784051, + "rescued": 4834, + "harmed": 1972, + "missed_rescue": 589, + "avoided_harm": 799, + "selected_both_correct": 13238, + "selected_both_wrong": 5500 + }, + { + "budget": 0.8, + "threshold": -6.614213943481445, + "operator": "ge", + "actual_fraction": 0.7337152255905623, + "metric": 0.7507302271502708, + "rescued": 4858, + "harmed": 1994, + "missed_rescue": 565, + "avoided_harm": 777, + "selected_both_correct": 13488, + "selected_both_wrong": 5533 + }, + { + "budget": 0.81, + "threshold": -6.680136680603027, + "operator": "ge", + "actual_fraction": 0.744377959901313, + "metric": 0.7505033604628081, + "rescued": 4880, + "harmed": 2024, + "missed_rescue": 543, + "avoided_harm": 747, + "selected_both_correct": 13784, + "selected_both_wrong": 5561 + }, + { + "budget": 0.82, + "threshold": -6.735454559326172, + "operator": "ge", + "actual_fraction": 0.7522048606187789, + "metric": 0.7503048521112781, + "rescued": 4897, + "harmed": 2048, + "missed_rescue": 526, + "avoided_harm": 723, + "selected_both_correct": 13998, + "selected_both_wrong": 5582 + }, + { + "budget": 0.83, + "threshold": -6.790655136108398, + "operator": "ge", + "actual_fraction": 0.7607123613986331, + "metric": 0.7503332104472109, + "rescued": 4923, + "harmed": 2073, + "missed_rescue": 500, + "avoided_harm": 698, + "selected_both_correct": 14223, + "selected_both_wrong": 5606 + }, + { + "budget": 0.84, + "threshold": -6.856695175170898, + "operator": "ge", + "actual_fraction": 0.770779570654794, + "metric": 0.749964552080084, + "rescued": 4937, + "harmed": 2100, + "missed_rescue": 486, + "avoided_harm": 671, + "selected_both_correct": 14511, + "selected_both_wrong": 5632 + }, + { + "budget": 0.85, + "threshold": -6.932329177856445, + "operator": "ge", + "actual_fraction": 0.7827467884184556, + "metric": 0.749964552080084, + "rescued": 4965, + "harmed": 2128, + "missed_rescue": 458, + "avoided_harm": 643, + "selected_both_correct": 14849, + "selected_both_wrong": 5660 + }, + { + "budget": 0.86, + "threshold": -6.999267578125, + "operator": "ge", + "actual_fraction": 0.793466239401072, + "metric": 0.7498511187363526, + "rescued": 4987, + "harmed": 2154, + "missed_rescue": 436, + "avoided_harm": 617, + "selected_both_correct": 15148, + "selected_both_wrong": 5691 + }, + { + "budget": 0.87, + "threshold": -7.081692695617676, + "operator": "ge", + "actual_fraction": 0.8059439072115249, + "metric": 0.7499361937441511, + "rescued": 5015, + "harmed": 2179, + "missed_rescue": 408, + "avoided_harm": 592, + "selected_both_correct": 15496, + "selected_both_wrong": 5730 + }, + { + "budget": 0.88, + "threshold": -7.155144214630127, + "operator": "ge", + "actual_fraction": 0.8164932081785441, + "metric": 0.7497093270566884, + "rescued": 5038, + "harmed": 2210, + "missed_rescue": 385, + "avoided_harm": 561, + "selected_both_correct": 15779, + "selected_both_wrong": 5765 + }, + { + "budget": 0.89, + "threshold": -7.227406978607178, + "operator": "ge", + "actual_fraction": 0.8270425091455633, + "metric": 0.7494257436973598, + "rescued": 5061, + "harmed": 2243, + "missed_rescue": 362, + "avoided_harm": 528, + "selected_both_correct": 16073, + "selected_both_wrong": 5787 + }, + { + "budget": 0.9, + "threshold": -7.308749198913574, + "operator": "ge", + "actual_fraction": 0.8367410600345971, + "metric": 0.749397385361427, + "rescued": 5087, + "harmed": 2270, + "missed_rescue": 336, + "avoided_harm": 501, + "selected_both_correct": 16348, + "selected_both_wrong": 5801 + }, + { + "budget": 0.91, + "threshold": -7.405246257781982, + "operator": "ge", + "actual_fraction": 0.8491903695091172, + "metric": 0.74883021864277, + "rescued": 5113, + "harmed": 2316, + "missed_rescue": 310, + "avoided_harm": 455, + "selected_both_correct": 16682, + "selected_both_wrong": 5834 + }, + { + "budget": 0.92, + "threshold": -7.509663105010986, + "operator": "gt", + "actual_fraction": 0.8618098289992343, + "metric": 0.7487167852990386, + "rescued": 5142, + "harmed": 2349, + "missed_rescue": 281, + "avoided_harm": 422, + "selected_both_correct": 17025, + "selected_both_wrong": 5874 + }, + { + "budget": 0.93, + "threshold": -7.625953674316406, + "operator": "ge", + "actual_fraction": 0.8745710801690156, + "metric": 0.7486884269631058, + "rescued": 5170, + "harmed": 2378, + "missed_rescue": 253, + "avoided_harm": 393, + "selected_both_correct": 17383, + "selected_both_wrong": 5909 + }, + { + "budget": 0.94, + "threshold": -7.762190818786621, + "operator": "ge", + "actual_fraction": 0.8879845730652526, + "metric": 0.7486884269631058, + "rescued": 5198, + "harmed": 2406, + "missed_rescue": 225, + "avoided_harm": 365, + "selected_both_correct": 17763, + "selected_both_wrong": 5946 + }, + { + "budget": 0.95, + "threshold": -7.891505241394043, + "operator": "ge", + "actual_fraction": 0.9000085075007799, + "metric": 0.7484332019397102, + "rescued": 5216, + "harmed": 2433, + "missed_rescue": 207, + "avoided_harm": 338, + "selected_both_correct": 18108, + "selected_both_wrong": 5980 + }, + { + "budget": 0.96, + "threshold": -8.049057960510254, + "operator": "ge", + "actual_fraction": 0.9125996086549641, + "metric": 0.7477809602132547, + "rescued": 5244, + "harmed": 2484, + "missed_rescue": 179, + "avoided_harm": 287, + "selected_both_correct": 18442, + "selected_both_wrong": 6011 + }, + { + "budget": 0.97, + "threshold": -8.248269081115723, + "operator": "ge", + "actual_fraction": 0.9269489266369849, + "metric": 0.7476391685335905, + "rescued": 5273, + "harmed": 2518, + "missed_rescue": 150, + "avoided_harm": 253, + "selected_both_correct": 18844, + "selected_both_wrong": 6052 + }, + { + "budget": 0.98, + "threshold": -8.543853759765625, + "operator": "ge", + "actual_fraction": 0.9459773700479256, + "metric": 0.7472988685023962, + "rescued": 5315, + "harmed": 2572, + "missed_rescue": 108, + "avoided_harm": 199, + "selected_both_correct": 19374, + "selected_both_wrong": 6097 + }, + { + "budget": 0.99, + "threshold": -8.96818733215332, + "operator": "ge", + "actual_fraction": 0.9658282052009188, + "metric": 0.7472421518305306, + "rescued": 5353, + "harmed": 2612, + "missed_rescue": 70, + "avoided_harm": 159, + "selected_both_correct": 19948, + "selected_both_wrong": 6145 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.7447182599325072, + "rescued": 5423, + "harmed": 2771, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 20838, + "selected_both_wrong": 6231 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 9505, + "rescuable": 1964, + "harmful": 1171, + "both_wrong": 1589, + "unit": "sample", + "total": 14229 + }, + "test": { + "both_correct": 20838, + "rescuable": 5423, + "harmful": 2771, + "both_wrong": 6231, + "unit": "sample", + "total": 35263 + } + }, + "small": { + "validation": 0.7502986857825568, + "test": 0.6695119530385957 + }, + "large": { + "validation": 0.8060299388572634, + "test": 0.7447182599325072 + }, + "learned_minus_confidence_test": 0.02850012761251175, + "learned_minus_small_test": 0.08331679097070588, + "learned_minus_large_test": 0.008110484076794378 + }, + { + "key": [ + "construction", + "yolo26x", + "Qwen3.8" + ], + "expert_display": "YOLO26x", + "run": "shared_12_yolo26x_Qwen3.8", + "validation_rows": 701, + "test_rows": 3004, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detection_count", + "has_detections", + "max_score", + "mean_score", + "detection_count_valid", + "has_detections_valid", + "max_score_valid", + "mean_score_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/yolo26x/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 93, + "validation_area_0_25": 0.17942553980644182, + "validation_point": { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2642878791936722, + "rescued": 288, + "harmed": 202, + "missed_rescue": 19, + "avoided_harm": 18, + "selected_both_correct": 2035, + "selected_both_wrong": 79 + }, + "test_point": { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9221038615179761, + "metric": 0.2597490004762105, + "rescued": 1180, + "harmed": 728, + "missed_rescue": 102, + "avoided_harm": 49, + "selected_both_correct": 8774, + "selected_both_wrong": 398 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": 1.0, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.1728507294858811, + "rescued": 0, + "harmed": 34, + "missed_rescue": 307, + "avoided_harm": 186, + "selected_both_correct": 498, + "selected_both_wrong": 0 + }, + { + "budget": 0.2, + "threshold": -0.0010652542114257812, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.1728507294858811, + "rescued": 0, + "harmed": 34, + "missed_rescue": 307, + "avoided_harm": 186, + "selected_both_correct": 526, + "selected_both_wrong": 0 + }, + { + "budget": 0.21, + "threshold": -0.0011425018310546875, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.17237009889855048, + "rescued": 0, + "harmed": 36, + "missed_rescue": 307, + "avoided_harm": 184, + "selected_both_correct": 552, + "selected_both_wrong": 0 + }, + { + "budget": 0.22, + "threshold": -0.0011968612670898438, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.17209204025345443, + "rescued": 0, + "harmed": 37, + "missed_rescue": 307, + "avoided_harm": 183, + "selected_both_correct": 578, + "selected_both_wrong": 1 + }, + { + "budget": 0.23, + "threshold": -0.0012493133544921875, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.1716297194027841, + "rescued": 0, + "harmed": 39, + "missed_rescue": 307, + "avoided_harm": 181, + "selected_both_correct": 604, + "selected_both_wrong": 1 + }, + { + "budget": 0.24, + "threshold": -0.0013513565063476562, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.17076888726505096, + "rescued": 0, + "harmed": 42, + "missed_rescue": 307, + "avoided_harm": 178, + "selected_both_correct": 629, + "selected_both_wrong": 1 + }, + { + "budget": 0.25, + "threshold": -0.0015010833740234375, + "operator": "gt", + "actual_fraction": 0.24821683309557774, + "metric": 0.17076888726505096, + "rescued": 0, + "harmed": 42, + "missed_rescue": 307, + "avoided_harm": 178, + "selected_both_correct": 652, + "selected_both_wrong": 2 + }, + { + "budget": 0.26, + "threshold": -0.00157928466796875, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.17010751160367527, + "rescued": 0, + "harmed": 45, + "missed_rescue": 307, + "avoided_harm": 175, + "selected_both_correct": 681, + "selected_both_wrong": 2 + }, + { + "budget": 0.27, + "threshold": -0.0017137527465820312, + "operator": "gt", + "actual_fraction": 0.268188302425107, + "metric": 0.17010751160367527, + "rescued": 0, + "harmed": 45, + "missed_rescue": 307, + "avoided_harm": 175, + "selected_both_correct": 705, + "selected_both_wrong": 2 + }, + { + "budget": 0.28, + "threshold": -0.0017614364624023438, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.16948188597804964, + "rescued": 0, + "harmed": 48, + "missed_rescue": 307, + "avoided_harm": 172, + "selected_both_correct": 734, + "selected_both_wrong": 2 + }, + { + "budget": 0.29, + "threshold": -0.0018463134765625, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.16948188597804964, + "rescued": 0, + "harmed": 48, + "missed_rescue": 307, + "avoided_harm": 172, + "selected_both_correct": 762, + "selected_both_wrong": 2 + }, + { + "budget": 0.3, + "threshold": -0.001934051513671875, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.16880965944961934, + "rescued": 0, + "harmed": 51, + "missed_rescue": 307, + "avoided_harm": 169, + "selected_both_correct": 787, + "selected_both_wrong": 2 + }, + { + "budget": 0.31, + "threshold": -0.0020427703857421875, + "operator": "gt", + "actual_fraction": 0.30813124108416545, + "metric": 0.16830460894456886, + "rescued": 0, + "harmed": 56, + "missed_rescue": 307, + "avoided_harm": 164, + "selected_both_correct": 806, + "selected_both_wrong": 2 + }, + { + "budget": 0.32, + "threshold": -0.0021495819091796875, + "operator": "gt", + "actual_fraction": 0.3181169757489301, + "metric": 0.16811053970066264, + "rescued": 0, + "harmed": 57, + "missed_rescue": 307, + "avoided_harm": 163, + "selected_both_correct": 833, + "selected_both_wrong": 2 + }, + { + "budget": 0.33, + "threshold": -0.002208709716796875, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.1671811299174608, + "rescued": 0, + "harmed": 61, + "missed_rescue": 307, + "avoided_harm": 159, + "selected_both_correct": 861, + "selected_both_wrong": 2 + }, + { + "budget": 0.34, + "threshold": -0.0023326873779296875, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.16645634059427164, + "rescued": 0, + "harmed": 64, + "missed_rescue": 307, + "avoided_harm": 156, + "selected_both_correct": 886, + "selected_both_wrong": 2 + }, + { + "budget": 0.35, + "threshold": -0.0024623870849609375, + "operator": "gt", + "actual_fraction": 0.34807417974322397, + "metric": 0.16600354762332126, + "rescued": 0, + "harmed": 66, + "missed_rescue": 307, + "avoided_harm": 154, + "selected_both_correct": 908, + "selected_both_wrong": 2 + }, + { + "budget": 0.36, + "threshold": -0.0025615692138671875, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.16502549737843858, + "rescued": 0, + "harmed": 70, + "missed_rescue": 307, + "avoided_harm": 150, + "selected_both_correct": 936, + "selected_both_wrong": 2 + }, + { + "budget": 0.37, + "threshold": -0.0026721954345703125, + "operator": "gt", + "actual_fraction": 0.3666191155492154, + "metric": 0.16467825104188744, + "rescued": 0, + "harmed": 72, + "missed_rescue": 307, + "avoided_harm": 148, + "selected_both_correct": 954, + "selected_both_wrong": 2 + }, + { + "budget": 0.38, + "threshold": -0.0028228759765625, + "operator": "gt", + "actual_fraction": 0.37660485021398005, + "metric": 0.16467825104188744, + "rescued": 0, + "harmed": 76, + "missed_rescue": 307, + "avoided_harm": 144, + "selected_both_correct": 978, + "selected_both_wrong": 2 + }, + { + "budget": 0.39, + "threshold": -0.0029354095458984375, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.16749463738508683, + "rescued": 1, + "harmed": 81, + "missed_rescue": 306, + "avoided_harm": 139, + "selected_both_correct": 1008, + "selected_both_wrong": 2 + }, + { + "budget": 0.4, + "threshold": -0.00307464599609375, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.16607474321791724, + "rescued": 1, + "harmed": 87, + "missed_rescue": 306, + "avoided_harm": 133, + "selected_both_correct": 1030, + "selected_both_wrong": 2 + }, + { + "budget": 0.41, + "threshold": -0.0033626556396484375, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.16514206572141965, + "rescued": 1, + "harmed": 90, + "missed_rescue": 306, + "avoided_harm": 130, + "selected_both_correct": 1055, + "selected_both_wrong": 2 + }, + { + "budget": 0.42, + "threshold": -0.0035381317138671875, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.16444252141114554, + "rescued": 1, + "harmed": 93, + "missed_rescue": 306, + "avoided_harm": 127, + "selected_both_correct": 1080, + "selected_both_wrong": 2 + }, + { + "budget": 0.43, + "threshold": -0.003765106201171875, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.1639139801849299, + "rescued": 1, + "harmed": 96, + "missed_rescue": 306, + "avoided_harm": 124, + "selected_both_correct": 1105, + "selected_both_wrong": 2 + }, + { + "budget": 0.44, + "threshold": -0.00392913818359375, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.16237039262040495, + "rescued": 1, + "harmed": 103, + "missed_rescue": 306, + "avoided_harm": 117, + "selected_both_correct": 1126, + "selected_both_wrong": 2 + }, + { + "budget": 0.45, + "threshold": -0.004085540771484375, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.16187473569847557, + "rescued": 1, + "harmed": 105, + "missed_rescue": 306, + "avoided_harm": 115, + "selected_both_correct": 1152, + "selected_both_wrong": 2 + }, + { + "budget": 0.46, + "threshold": -0.004364013671875, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.1640341119762699, + "rescued": 2, + "harmed": 110, + "missed_rescue": 305, + "avoided_harm": 110, + "selected_both_correct": 1174, + "selected_both_wrong": 2 + }, + { + "budget": 0.47, + "threshold": -0.004680633544921875, + "operator": "gt", + "actual_fraction": 0.46790299572039945, + "metric": 0.16656779979633995, + "rescued": 3, + "harmed": 113, + "missed_rescue": 304, + "avoided_harm": 107, + "selected_both_correct": 1194, + "selected_both_wrong": 2 + }, + { + "budget": 0.48, + "threshold": -0.00490570068359375, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.16532395129425315, + "rescued": 3, + "harmed": 119, + "missed_rescue": 304, + "avoided_harm": 101, + "selected_both_correct": 1220, + "selected_both_wrong": 2 + }, + { + "budget": 0.49, + "threshold": -0.005039215087890625, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.16467625831496033, + "rescued": 5, + "harmed": 123, + "missed_rescue": 302, + "avoided_harm": 97, + "selected_both_correct": 1241, + "selected_both_wrong": 3 + }, + { + "budget": 0.5, + "threshold": -0.005321502685546875, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.16569589650413527, + "rescued": 11, + "harmed": 128, + "missed_rescue": 296, + "avoided_harm": 92, + "selected_both_correct": 1257, + "selected_both_wrong": 4 + }, + { + "budget": 0.51, + "threshold": -0.005619049072265625, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.1671927745457157, + "rescued": 18, + "harmed": 129, + "missed_rescue": 289, + "avoided_harm": 91, + "selected_both_correct": 1276, + "selected_both_wrong": 5 + }, + { + "budget": 0.52, + "threshold": -0.00586700439453125, + "operator": "gt", + "actual_fraction": 0.5178316690442225, + "metric": 0.16789087940773334, + "rescued": 23, + "harmed": 131, + "missed_rescue": 284, + "avoided_harm": 89, + "selected_both_correct": 1293, + "selected_both_wrong": 5 + }, + { + "budget": 0.53, + "threshold": -0.00605010986328125, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.16946103210148153, + "rescued": 30, + "harmed": 132, + "missed_rescue": 277, + "avoided_harm": 88, + "selected_both_correct": 1316, + "selected_both_wrong": 6 + }, + { + "budget": 0.54, + "threshold": -0.006439208984375, + "operator": "gt", + "actual_fraction": 0.5378031383737518, + "metric": 0.17052688690619724, + "rescued": 35, + "harmed": 133, + "missed_rescue": 272, + "avoided_harm": 87, + "selected_both_correct": 1334, + "selected_both_wrong": 6 + }, + { + "budget": 0.55, + "threshold": -0.006641387939453125, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.17103667850460152, + "rescued": 42, + "harmed": 134, + "missed_rescue": 265, + "avoided_harm": 86, + "selected_both_correct": 1358, + "selected_both_wrong": 6 + }, + { + "budget": 0.56, + "threshold": -0.0070648193359375, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.17291017019862398, + "rescued": 49, + "harmed": 135, + "missed_rescue": 258, + "avoided_harm": 85, + "selected_both_correct": 1376, + "selected_both_wrong": 8 + }, + { + "budget": 0.57, + "threshold": -0.007518768310546875, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.17468909977784217, + "rescued": 55, + "harmed": 135, + "missed_rescue": 252, + "avoided_harm": 85, + "selected_both_correct": 1397, + "selected_both_wrong": 9 + }, + { + "budget": 0.58, + "threshold": -0.00818634033203125, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.1754433851238893, + "rescued": 60, + "harmed": 136, + "missed_rescue": 247, + "avoided_harm": 84, + "selected_both_correct": 1416, + "selected_both_wrong": 12 + }, + { + "budget": 0.59, + "threshold": -0.00878143310546875, + "operator": "gt", + "actual_fraction": 0.5877318116975749, + "metric": 0.17567949038614306, + "rescued": 65, + "harmed": 140, + "missed_rescue": 242, + "avoided_harm": 80, + "selected_both_correct": 1430, + "selected_both_wrong": 13 + }, + { + "budget": 0.6, + "threshold": -0.00955963134765625, + "operator": "gt", + "actual_fraction": 0.5977175463623395, + "metric": 0.17726242688944494, + "rescued": 73, + "harmed": 142, + "missed_rescue": 234, + "avoided_harm": 78, + "selected_both_correct": 1448, + "selected_both_wrong": 13 + }, + { + "budget": 0.61, + "threshold": -0.0099334716796875, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.17927753989081613, + "rescued": 81, + "harmed": 143, + "missed_rescue": 226, + "avoided_harm": 77, + "selected_both_correct": 1471, + "selected_both_wrong": 13 + }, + { + "budget": 0.62, + "threshold": -0.010650634765625, + "operator": "gt", + "actual_fraction": 0.6176890156918687, + "metric": 0.1803426895170312, + "rescued": 86, + "harmed": 144, + "missed_rescue": 221, + "avoided_harm": 76, + "selected_both_correct": 1489, + "selected_both_wrong": 13 + }, + { + "budget": 0.63, + "threshold": -0.01129150390625, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.18256297675338154, + "rescued": 94, + "harmed": 145, + "missed_rescue": 213, + "avoided_harm": 75, + "selected_both_correct": 1509, + "selected_both_wrong": 16 + }, + { + "budget": 0.64, + "threshold": -0.011871337890625, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.18741054744283442, + "rescued": 103, + "harmed": 149, + "missed_rescue": 204, + "avoided_harm": 71, + "selected_both_correct": 1524, + "selected_both_wrong": 16 + }, + { + "budget": 0.65, + "threshold": -0.0125274658203125, + "operator": "gt", + "actual_fraction": 0.6476462196861626, + "metric": 0.18874453577558464, + "rescued": 109, + "harmed": 150, + "missed_rescue": 198, + "avoided_harm": 70, + "selected_both_correct": 1540, + "selected_both_wrong": 17 + }, + { + "budget": 0.66, + "threshold": -0.012969970703125, + "operator": "gt", + "actual_fraction": 0.6562054208273894, + "metric": 0.18973030032445837, + "rescued": 114, + "harmed": 152, + "missed_rescue": 193, + "avoided_harm": 68, + "selected_both_correct": 1556, + "selected_both_wrong": 18 + }, + { + "budget": 0.67, + "threshold": -0.01363372802734375, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.1910322331722189, + "rescued": 122, + "harmed": 157, + "missed_rescue": 185, + "avoided_harm": 63, + "selected_both_correct": 1576, + "selected_both_wrong": 21 + }, + { + "budget": 0.68, + "threshold": -0.01473236083984375, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.19273815380416676, + "rescued": 128, + "harmed": 158, + "missed_rescue": 179, + "avoided_harm": 62, + "selected_both_correct": 1595, + "selected_both_wrong": 23 + }, + { + "budget": 0.69, + "threshold": -0.015716552734375, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.19362404817095658, + "rescued": 133, + "harmed": 161, + "missed_rescue": 174, + "avoided_harm": 59, + "selected_both_correct": 1612, + "selected_both_wrong": 26 + }, + { + "budget": 0.7, + "threshold": -0.017578125, + "operator": "gt", + "actual_fraction": 0.6975748930099858, + "metric": 0.19489832797986031, + "rescued": 138, + "harmed": 163, + "missed_rescue": 169, + "avoided_harm": 57, + "selected_both_correct": 1626, + "selected_both_wrong": 29 + }, + { + "budget": 0.71, + "threshold": -0.0189056396484375, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.19709243659719292, + "rescued": 147, + "harmed": 166, + "missed_rescue": 160, + "avoided_harm": 54, + "selected_both_correct": 1646, + "selected_both_wrong": 29 + }, + { + "budget": 0.72, + "threshold": -0.019866943359375, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.20000323882160753, + "rescued": 155, + "harmed": 166, + "missed_rescue": 152, + "avoided_harm": 54, + "selected_both_correct": 1666, + "selected_both_wrong": 29 + }, + { + "budget": 0.73, + "threshold": -0.020965576171875, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.2015690267335004, + "rescued": 160, + "harmed": 167, + "missed_rescue": 147, + "avoided_harm": 53, + "selected_both_correct": 1684, + "selected_both_wrong": 33 + }, + { + "budget": 0.74, + "threshold": -0.021453857421875, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.2034984942593638, + "rescued": 166, + "harmed": 168, + "missed_rescue": 141, + "avoided_harm": 52, + "selected_both_correct": 1703, + "selected_both_wrong": 35 + }, + { + "budget": 0.75, + "threshold": -0.0248870849609375, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.20679879821670866, + "rescued": 176, + "harmed": 170, + "missed_rescue": 131, + "avoided_harm": 50, + "selected_both_correct": 1718, + "selected_both_wrong": 36 + }, + { + "budget": 0.76, + "threshold": -0.0269622802734375, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.20972162293572166, + "rescued": 184, + "harmed": 171, + "missed_rescue": 123, + "avoided_harm": 49, + "selected_both_correct": 1735, + "selected_both_wrong": 38 + }, + { + "budget": 0.77, + "threshold": -0.0282135009765625, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.2126226551226551, + "rescued": 191, + "harmed": 174, + "missed_rescue": 116, + "avoided_harm": 46, + "selected_both_correct": 1752, + "selected_both_wrong": 39 + }, + { + "budget": 0.78, + "threshold": -0.0299835205078125, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.21516614155218836, + "rescued": 198, + "harmed": 175, + "missed_rescue": 109, + "avoided_harm": 45, + "selected_both_correct": 1771, + "selected_both_wrong": 40 + }, + { + "budget": 0.79, + "threshold": -0.033966064453125, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.2162880860385455, + "rescued": 204, + "harmed": 177, + "missed_rescue": 103, + "avoided_harm": 43, + "selected_both_correct": 1788, + "selected_both_wrong": 43 + }, + { + "budget": 0.8, + "threshold": -0.037200927734375, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.21779120256650406, + "rescued": 209, + "harmed": 179, + "missed_rescue": 98, + "avoided_harm": 41, + "selected_both_correct": 1806, + "selected_both_wrong": 46 + }, + { + "budget": 0.81, + "threshold": -0.04132080078125, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.2201843117420812, + "rescued": 216, + "harmed": 182, + "missed_rescue": 91, + "avoided_harm": 38, + "selected_both_correct": 1820, + "selected_both_wrong": 50 + }, + { + "budget": 0.82, + "threshold": -0.0455322265625, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.22188537527520574, + "rescued": 222, + "harmed": 184, + "missed_rescue": 85, + "avoided_harm": 36, + "selected_both_correct": 1838, + "selected_both_wrong": 52 + }, + { + "budget": 0.83, + "threshold": -0.051849365234375, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.2260242392475367, + "rescued": 231, + "harmed": 187, + "missed_rescue": 76, + "avoided_harm": 33, + "selected_both_correct": 1854, + "selected_both_wrong": 52 + }, + { + "budget": 0.84, + "threshold": -0.06878662109375, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.2282665192342907, + "rescued": 238, + "harmed": 190, + "missed_rescue": 69, + "avoided_harm": 30, + "selected_both_correct": 1869, + "selected_both_wrong": 55 + }, + { + "budget": 0.85, + "threshold": -0.08477783203125, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.23641538775702553, + "rescued": 246, + "harmed": 193, + "missed_rescue": 61, + "avoided_harm": 27, + "selected_both_correct": 1884, + "selected_both_wrong": 57 + }, + { + "budget": 0.86, + "threshold": -0.09912109375, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.2387681003514125, + "rescued": 251, + "harmed": 193, + "missed_rescue": 56, + "avoided_harm": 27, + "selected_both_correct": 1904, + "selected_both_wrong": 60 + }, + { + "budget": 0.87, + "threshold": -0.1298828125, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.24246700995212897, + "rescued": 258, + "harmed": 193, + "missed_rescue": 49, + "avoided_harm": 27, + "selected_both_correct": 1923, + "selected_both_wrong": 62 + }, + { + "budget": 0.88, + "threshold": -0.1441650390625, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.2423696527366133, + "rescued": 259, + "harmed": 197, + "missed_rescue": 48, + "avoided_harm": 23, + "selected_both_correct": 1941, + "selected_both_wrong": 67 + }, + { + "budget": 0.89, + "threshold": -0.182373046875, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.24478740599615373, + "rescued": 266, + "harmed": 199, + "missed_rescue": 41, + "avoided_harm": 21, + "selected_both_correct": 1959, + "selected_both_wrong": 68 + }, + { + "budget": 0.9, + "threshold": -0.2049560546875, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.25736315359477124, + "rescued": 273, + "harmed": 199, + "missed_rescue": 34, + "avoided_harm": 21, + "selected_both_correct": 1977, + "selected_both_wrong": 71 + }, + { + "budget": 0.91, + "threshold": -0.25830078125, + "operator": "gt", + "actual_fraction": 0.9072753209700428, + "metric": 0.2597551406926407, + "rescued": 278, + "harmed": 200, + "missed_rescue": 29, + "avoided_harm": 20, + "selected_both_correct": 1992, + "selected_both_wrong": 74 + }, + { + "budget": 0.92, + "threshold": -0.32080078125, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.2629241291812946, + "rescued": 284, + "harmed": 201, + "missed_rescue": 23, + "avoided_harm": 19, + "selected_both_correct": 2013, + "selected_both_wrong": 78 + }, + { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2642878791936722, + "rescued": 288, + "harmed": 202, + "missed_rescue": 19, + "avoided_harm": 18, + "selected_both_correct": 2035, + "selected_both_wrong": 79 + }, + { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.2639590174546567, + "rescued": 290, + "harmed": 203, + "missed_rescue": 17, + "avoided_harm": 17, + "selected_both_correct": 2057, + "selected_both_wrong": 82 + }, + { + "budget": 0.95, + "threshold": -0.611328125, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.26132796745180287, + "rescued": 293, + "harmed": 207, + "missed_rescue": 14, + "avoided_harm": 13, + "selected_both_correct": 2078, + "selected_both_wrong": 82 + }, + { + "budget": 0.96, + "threshold": -0.70068359375, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.2584345323365104, + "rescued": 296, + "harmed": 212, + "missed_rescue": 11, + "avoided_harm": 8, + "selected_both_correct": 2096, + "selected_both_wrong": 84 + }, + { + "budget": 0.97, + "threshold": -0.75048828125, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.2503146324574896, + "rescued": 299, + "harmed": 215, + "missed_rescue": 8, + "avoided_harm": 5, + "selected_both_correct": 2116, + "selected_both_wrong": 86 + }, + { + "budget": 0.98, + "threshold": -0.7841796875, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.2545437366865938, + "rescued": 302, + "harmed": 216, + "missed_rescue": 5, + "avoided_harm": 4, + "selected_both_correct": 2139, + "selected_both_wrong": 87 + }, + { + "budget": 0.99, + "threshold": -0.85107421875, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.2541444987255433, + "rescued": 304, + "harmed": 218, + "missed_rescue": 3, + "avoided_harm": 2, + "selected_both_correct": 2163, + "selected_both_wrong": 87 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.25448626222520154, + "rescued": 307, + "harmed": 220, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 2189, + "selected_both_wrong": 88 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": 1.0, + "operator": "ge", + "actual_fraction": 0.17942743009320905, + "metric": 0.1784481207469488, + "rescued": 4, + "harmed": 94, + "missed_rescue": 1278, + "avoided_harm": 683, + "selected_both_correct": 2057, + "selected_both_wrong": 1 + }, + { + "budget": 0.2, + "threshold": -0.0010652542114257812, + "operator": "ge", + "actual_fraction": 0.19340878828229027, + "metric": 0.1780928651788194, + "rescued": 4, + "harmed": 99, + "missed_rescue": 1278, + "avoided_harm": 678, + "selected_both_correct": 2220, + "selected_both_wrong": 1 + }, + { + "budget": 0.21, + "threshold": -0.0011425018310546875, + "operator": "ge", + "actual_fraction": 0.20306258322237017, + "metric": 0.17829989216687975, + "rescued": 5, + "harmed": 108, + "missed_rescue": 1277, + "avoided_harm": 669, + "selected_both_correct": 2325, + "selected_both_wrong": 2 + }, + { + "budget": 0.22, + "threshold": -0.0011968612670898438, + "operator": "ge", + "actual_fraction": 0.20838881491344874, + "metric": 0.1779854683890517, + "rescued": 5, + "harmed": 112, + "missed_rescue": 1277, + "avoided_harm": 665, + "selected_both_correct": 2385, + "selected_both_wrong": 2 + }, + { + "budget": 0.23, + "threshold": -0.0012493133544921875, + "operator": "ge", + "actual_fraction": 0.21537949400798934, + "metric": 0.17917657728013692, + "rescued": 6, + "harmed": 118, + "missed_rescue": 1276, + "avoided_harm": 659, + "selected_both_correct": 2462, + "selected_both_wrong": 2 + }, + { + "budget": 0.24, + "threshold": -0.0013513565063476562, + "operator": "ge", + "actual_fraction": 0.23135818908122505, + "metric": 0.17803687432776102, + "rescued": 6, + "harmed": 133, + "missed_rescue": 1276, + "avoided_harm": 644, + "selected_both_correct": 2638, + "selected_both_wrong": 3 + }, + { + "budget": 0.25, + "threshold": -0.0015010833740234375, + "operator": "gt", + "actual_fraction": 0.24800266311584554, + "metric": 0.18026802885144627, + "rescued": 10, + "harmed": 153, + "missed_rescue": 1272, + "avoided_harm": 624, + "selected_both_correct": 2814, + "selected_both_wrong": 3 + }, + { + "budget": 0.26, + "threshold": -0.00157928466796875, + "operator": "ge", + "actual_fraction": 0.2589880159786951, + "metric": 0.18119827871024874, + "rescued": 11, + "harmed": 163, + "missed_rescue": 1271, + "avoided_harm": 614, + "selected_both_correct": 2935, + "selected_both_wrong": 3 + }, + { + "budget": 0.27, + "threshold": -0.0017137527465820312, + "operator": "gt", + "actual_fraction": 0.27430093209054596, + "metric": 0.17963665085316138, + "rescued": 11, + "harmed": 185, + "missed_rescue": 1271, + "avoided_harm": 592, + "selected_both_correct": 3097, + "selected_both_wrong": 3 + }, + { + "budget": 0.28, + "threshold": -0.0017614364624023438, + "operator": "ge", + "actual_fraction": 0.281624500665779, + "metric": 0.17924095839307458, + "rescued": 11, + "harmed": 191, + "missed_rescue": 1271, + "avoided_harm": 586, + "selected_both_correct": 3179, + "selected_both_wrong": 3 + }, + { + "budget": 0.29, + "threshold": -0.0018463134765625, + "operator": "ge", + "actual_fraction": 0.28761651131824234, + "metric": 0.1789157174618718, + "rescued": 11, + "harmed": 196, + "missed_rescue": 1271, + "avoided_harm": 581, + "selected_both_correct": 3246, + "selected_both_wrong": 3 + }, + { + "budget": 0.3, + "threshold": -0.001934051513671875, + "operator": "ge", + "actual_fraction": 0.2969374167776298, + "metric": 0.1787206845800119, + "rescued": 11, + "harmed": 199, + "missed_rescue": 1271, + "avoided_harm": 578, + "selected_both_correct": 3355, + "selected_both_wrong": 3 + }, + { + "budget": 0.31, + "threshold": -0.0020427703857421875, + "operator": "gt", + "actual_fraction": 0.30459387483355527, + "metric": 0.17813242189706796, + "rescued": 11, + "harmed": 207, + "missed_rescue": 1271, + "avoided_harm": 570, + "selected_both_correct": 3438, + "selected_both_wrong": 4 + }, + { + "budget": 0.32, + "threshold": -0.0021495819091796875, + "operator": "gt", + "actual_fraction": 0.3129161118508655, + "metric": 0.17772000842137498, + "rescued": 12, + "harmed": 218, + "missed_rescue": 1270, + "avoided_harm": 559, + "selected_both_correct": 3526, + "selected_both_wrong": 4 + }, + { + "budget": 0.33, + "threshold": -0.002208709716796875, + "operator": "ge", + "actual_fraction": 0.31990679094540614, + "metric": 0.17791259734473341, + "rescued": 13, + "harmed": 227, + "missed_rescue": 1269, + "avoided_harm": 550, + "selected_both_correct": 3600, + "selected_both_wrong": 4 + }, + { + "budget": 0.34, + "threshold": -0.0023326873779296875, + "operator": "ge", + "actual_fraction": 0.3322237017310253, + "metric": 0.1789653934599337, + "rescued": 15, + "harmed": 238, + "missed_rescue": 1267, + "avoided_harm": 539, + "selected_both_correct": 3735, + "selected_both_wrong": 4 + }, + { + "budget": 0.35, + "threshold": -0.0024623870849609375, + "operator": "gt", + "actual_fraction": 0.34054593874833555, + "metric": 0.18042682200794885, + "rescued": 16, + "harmed": 249, + "missed_rescue": 1266, + "avoided_harm": 528, + "selected_both_correct": 3822, + "selected_both_wrong": 5 + }, + { + "budget": 0.36, + "threshold": -0.0025615692138671875, + "operator": "ge", + "actual_fraction": 0.3488681757656458, + "metric": 0.17962877325912951, + "rescued": 16, + "harmed": 260, + "missed_rescue": 1266, + "avoided_harm": 517, + "selected_both_correct": 3910, + "selected_both_wrong": 6 + }, + { + "budget": 0.37, + "threshold": -0.0026721954345703125, + "operator": "gt", + "actual_fraction": 0.3581890812250333, + "metric": 0.18158304685510065, + "rescued": 18, + "harmed": 271, + "missed_rescue": 1264, + "avoided_harm": 506, + "selected_both_correct": 4008, + "selected_both_wrong": 7 + }, + { + "budget": 0.38, + "threshold": -0.0028228759765625, + "operator": "gt", + "actual_fraction": 0.37017310252996005, + "metric": 0.18069239723097547, + "rescued": 18, + "harmed": 285, + "missed_rescue": 1264, + "avoided_harm": 492, + "selected_both_correct": 4138, + "selected_both_wrong": 7 + }, + { + "budget": 0.39, + "threshold": -0.0029354095458984375, + "operator": "ge", + "actual_fraction": 0.37683089214380827, + "metric": 0.1812845803851709, + "rescued": 20, + "harmed": 292, + "missed_rescue": 1262, + "avoided_harm": 485, + "selected_both_correct": 4209, + "selected_both_wrong": 7 + }, + { + "budget": 0.4, + "threshold": -0.00307464599609375, + "operator": "ge", + "actual_fraction": 0.38348868175765644, + "metric": 0.1824781664689562, + "rescued": 21, + "harmed": 301, + "missed_rescue": 1261, + "avoided_harm": 476, + "selected_both_correct": 4279, + "selected_both_wrong": 7 + }, + { + "budget": 0.41, + "threshold": -0.0033626556396484375, + "operator": "ge", + "actual_fraction": 0.4021304926764314, + "metric": 0.18220594501399923, + "rescued": 23, + "harmed": 321, + "missed_rescue": 1259, + "avoided_harm": 456, + "selected_both_correct": 4480, + "selected_both_wrong": 8 + }, + { + "budget": 0.42, + "threshold": -0.0035381317138671875, + "operator": "ge", + "actual_fraction": 0.41311584553928093, + "metric": 0.18162498930772586, + "rescued": 23, + "harmed": 329, + "missed_rescue": 1259, + "avoided_harm": 448, + "selected_both_correct": 4603, + "selected_both_wrong": 9 + }, + { + "budget": 0.43, + "threshold": -0.003765106201171875, + "operator": "ge", + "actual_fraction": 0.42676431424766975, + "metric": 0.18066434967152634, + "rescued": 23, + "harmed": 351, + "missed_rescue": 1259, + "avoided_harm": 426, + "selected_both_correct": 4745, + "selected_both_wrong": 9 + }, + { + "budget": 0.44, + "threshold": -0.00392913818359375, + "operator": "ge", + "actual_fraction": 0.4324234354194407, + "metric": 0.1803913789289444, + "rescued": 23, + "harmed": 355, + "missed_rescue": 1259, + "avoided_harm": 422, + "selected_both_correct": 4809, + "selected_both_wrong": 9 + }, + { + "budget": 0.45, + "threshold": -0.004085540771484375, + "operator": "ge", + "actual_fraction": 0.44107856191744343, + "metric": 0.17959468902865128, + "rescued": 23, + "harmed": 368, + "missed_rescue": 1259, + "avoided_harm": 409, + "selected_both_correct": 4900, + "selected_both_wrong": 9 + }, + { + "budget": 0.46, + "threshold": -0.004364013671875, + "operator": "ge", + "actual_fraction": 0.45772303595206393, + "metric": 0.17863383514615866, + "rescued": 24, + "harmed": 387, + "missed_rescue": 1258, + "avoided_harm": 390, + "selected_both_correct": 5078, + "selected_both_wrong": 11 + }, + { + "budget": 0.47, + "threshold": -0.004680633544921875, + "operator": "gt", + "actual_fraction": 0.4717043941411451, + "metric": 0.17957308538379646, + "rescued": 26, + "harmed": 405, + "missed_rescue": 1256, + "avoided_harm": 372, + "selected_both_correct": 5225, + "selected_both_wrong": 12 + }, + { + "budget": 0.48, + "threshold": -0.00490570068359375, + "operator": "ge", + "actual_fraction": 0.4826897470039947, + "metric": 0.17910245145856155, + "rescued": 27, + "harmed": 417, + "missed_rescue": 1255, + "avoided_harm": 360, + "selected_both_correct": 5344, + "selected_both_wrong": 12 + }, + { + "budget": 0.49, + "threshold": -0.005039215087890625, + "operator": "ge", + "actual_fraction": 0.4896804260985353, + "metric": 0.17871833907402393, + "rescued": 29, + "harmed": 425, + "missed_rescue": 1253, + "avoided_harm": 352, + "selected_both_correct": 5416, + "selected_both_wrong": 14 + }, + { + "budget": 0.5, + "threshold": -0.005321502685546875, + "operator": "ge", + "actual_fraction": 0.5013315579227696, + "metric": 0.18015785955296285, + "rescued": 57, + "harmed": 431, + "missed_rescue": 1225, + "avoided_harm": 346, + "selected_both_correct": 5516, + "selected_both_wrong": 20 + }, + { + "budget": 0.51, + "threshold": -0.005619049072265625, + "operator": "ge", + "actual_fraction": 0.5116511318242344, + "metric": 0.1810878167493375, + "rescued": 76, + "harmed": 435, + "missed_rescue": 1206, + "avoided_harm": 342, + "selected_both_correct": 5606, + "selected_both_wrong": 31 + }, + { + "budget": 0.52, + "threshold": -0.00586700439453125, + "operator": "gt", + "actual_fraction": 0.5183089214380826, + "metric": 0.18177962418520882, + "rescued": 92, + "harmed": 439, + "missed_rescue": 1190, + "avoided_harm": 338, + "selected_both_correct": 5662, + "selected_both_wrong": 35 + }, + { + "budget": 0.53, + "threshold": -0.00605010986328125, + "operator": "ge", + "actual_fraction": 0.5252996005326231, + "metric": 0.18246323539519374, + "rescued": 109, + "harmed": 445, + "missed_rescue": 1173, + "avoided_harm": 332, + "selected_both_correct": 5719, + "selected_both_wrong": 39 + }, + { + "budget": 0.54, + "threshold": -0.006439208984375, + "operator": "gt", + "actual_fraction": 0.5396138482023968, + "metric": 0.1839876166906015, + "rescued": 150, + "harmed": 461, + "missed_rescue": 1132, + "avoided_harm": 316, + "selected_both_correct": 5829, + "selected_both_wrong": 44 + }, + { + "budget": 0.55, + "threshold": -0.006641387939453125, + "operator": "ge", + "actual_fraction": 0.5452729693741678, + "metric": 0.1846674213907964, + "rescued": 164, + "harmed": 464, + "missed_rescue": 1118, + "avoided_harm": 313, + "selected_both_correct": 5875, + "selected_both_wrong": 49 + }, + { + "budget": 0.56, + "threshold": -0.0070648193359375, + "operator": "ge", + "actual_fraction": 0.559254327563249, + "metric": 0.18676294233791663, + "rescued": 196, + "harmed": 473, + "missed_rescue": 1086, + "avoided_harm": 304, + "selected_both_correct": 5990, + "selected_both_wrong": 61 + }, + { + "budget": 0.57, + "threshold": -0.007518768310546875, + "operator": "ge", + "actual_fraction": 0.572237017310253, + "metric": 0.18938663221175195, + "rescued": 230, + "harmed": 483, + "missed_rescue": 1052, + "avoided_harm": 294, + "selected_both_correct": 6093, + "selected_both_wrong": 70 + }, + { + "budget": 0.58, + "threshold": -0.00818634033203125, + "operator": "ge", + "actual_fraction": 0.5862183754993342, + "metric": 0.1909571002676625, + "rescued": 267, + "harmed": 496, + "missed_rescue": 1015, + "avoided_harm": 281, + "selected_both_correct": 6208, + "selected_both_wrong": 73 + }, + { + "budget": 0.59, + "threshold": -0.00878143310546875, + "operator": "gt", + "actual_fraction": 0.5998668442077231, + "metric": 0.1934300101790686, + "rescued": 299, + "harmed": 503, + "missed_rescue": 983, + "avoided_harm": 274, + "selected_both_correct": 6315, + "selected_both_wrong": 91 + }, + { + "budget": 0.6, + "threshold": -0.00955963134765625, + "operator": "gt", + "actual_fraction": 0.6191744340878829, + "metric": 0.1973701682130997, + "rescued": 350, + "harmed": 520, + "missed_rescue": 932, + "avoided_harm": 257, + "selected_both_correct": 6465, + "selected_both_wrong": 105 + }, + { + "budget": 0.61, + "threshold": -0.0099334716796875, + "operator": "ge", + "actual_fraction": 0.6264980026631158, + "metric": 0.1986727924998457, + "rescued": 370, + "harmed": 523, + "missed_rescue": 912, + "avoided_harm": 254, + "selected_both_correct": 6527, + "selected_both_wrong": 108 + }, + { + "budget": 0.62, + "threshold": -0.010650634765625, + "operator": "gt", + "actual_fraction": 0.6384820239680427, + "metric": 0.20186289191698192, + "rescued": 402, + "harmed": 532, + "missed_rescue": 880, + "avoided_harm": 245, + "selected_both_correct": 6621, + "selected_both_wrong": 117 + }, + { + "budget": 0.63, + "threshold": -0.01129150390625, + "operator": "ge", + "actual_fraction": 0.6461384820239681, + "metric": 0.20246031058520178, + "rescued": 425, + "harmed": 537, + "missed_rescue": 857, + "avoided_harm": 240, + "selected_both_correct": 6682, + "selected_both_wrong": 120 + }, + { + "budget": 0.64, + "threshold": -0.011871337890625, + "operator": "ge", + "actual_fraction": 0.653462050599201, + "metric": 0.20349944137622245, + "rescued": 444, + "harmed": 539, + "missed_rescue": 838, + "avoided_harm": 238, + "selected_both_correct": 6744, + "selected_both_wrong": 125 + }, + { + "budget": 0.65, + "threshold": -0.0125274658203125, + "operator": "gt", + "actual_fraction": 0.6597869507323568, + "metric": 0.20442990090722246, + "rescued": 461, + "harmed": 545, + "missed_rescue": 821, + "avoided_harm": 232, + "selected_both_correct": 6792, + "selected_both_wrong": 130 + }, + { + "budget": 0.66, + "threshold": -0.012969970703125, + "operator": "gt", + "actual_fraction": 0.6661118508655126, + "metric": 0.2044993942500819, + "rescued": 480, + "harmed": 556, + "missed_rescue": 802, + "avoided_harm": 221, + "selected_both_correct": 6834, + "selected_both_wrong": 134 + }, + { + "budget": 0.67, + "threshold": -0.01363372802734375, + "operator": "ge", + "actual_fraction": 0.6734354194407457, + "metric": 0.2064379542733177, + "rescued": 500, + "harmed": 561, + "missed_rescue": 782, + "avoided_harm": 216, + "selected_both_correct": 6889, + "selected_both_wrong": 142 + }, + { + "budget": 0.68, + "threshold": -0.01473236083984375, + "operator": "ge", + "actual_fraction": 0.6820905459387483, + "metric": 0.2065831206227621, + "rescued": 523, + "harmed": 566, + "missed_rescue": 759, + "avoided_harm": 211, + "selected_both_correct": 6958, + "selected_both_wrong": 149 + }, + { + "budget": 0.69, + "threshold": -0.015716552734375, + "operator": "ge", + "actual_fraction": 0.6927430093209055, + "metric": 0.20802802394105854, + "rescued": 553, + "harmed": 577, + "missed_rescue": 729, + "avoided_harm": 200, + "selected_both_correct": 7035, + "selected_both_wrong": 159 + }, + { + "budget": 0.7, + "threshold": -0.017578125, + "operator": "gt", + "actual_fraction": 0.7090545938748336, + "metric": 0.21169680596278434, + "rescued": 602, + "harmed": 584, + "missed_rescue": 680, + "avoided_harm": 193, + "selected_both_correct": 7159, + "selected_both_wrong": 175 + }, + { + "budget": 0.71, + "threshold": -0.0189056396484375, + "operator": "ge", + "actual_fraction": 0.7170439414114514, + "metric": 0.2139280527103987, + "rescued": 625, + "harmed": 588, + "missed_rescue": 657, + "avoided_harm": 189, + "selected_both_correct": 7220, + "selected_both_wrong": 183 + }, + { + "budget": 0.72, + "threshold": -0.019866943359375, + "operator": "ge", + "actual_fraction": 0.7243675099866844, + "metric": 0.2146900231025185, + "rescued": 647, + "harmed": 594, + "missed_rescue": 635, + "avoided_harm": 183, + "selected_both_correct": 7268, + "selected_both_wrong": 195 + }, + { + "budget": 0.73, + "threshold": -0.020965576171875, + "operator": "ge", + "actual_fraction": 0.7330226364846871, + "metric": 0.21653561577692484, + "rescued": 670, + "harmed": 599, + "missed_rescue": 612, + "avoided_harm": 178, + "selected_both_correct": 7338, + "selected_both_wrong": 201 + }, + { + "budget": 0.74, + "threshold": -0.021453857421875, + "operator": "ge", + "actual_fraction": 0.7363515312916112, + "metric": 0.21721689648133474, + "rescued": 681, + "harmed": 601, + "missed_rescue": 601, + "avoided_harm": 176, + "selected_both_correct": 7365, + "selected_both_wrong": 201 + }, + { + "budget": 0.75, + "threshold": -0.0248870849609375, + "operator": "ge", + "actual_fraction": 0.7486684420772304, + "metric": 0.22103445955201576, + "rescued": 718, + "harmed": 610, + "missed_rescue": 564, + "avoided_harm": 167, + "selected_both_correct": 7460, + "selected_both_wrong": 208 + }, + { + "budget": 0.76, + "threshold": -0.0269622802734375, + "operator": "ge", + "actual_fraction": 0.757323568575233, + "metric": 0.22340134248367294, + "rescued": 744, + "harmed": 618, + "missed_rescue": 538, + "avoided_harm": 159, + "selected_both_correct": 7522, + "selected_both_wrong": 216 + }, + { + "budget": 0.77, + "threshold": -0.0282135009765625, + "operator": "ge", + "actual_fraction": 0.7639813581890812, + "metric": 0.2269322568854726, + "rescued": 767, + "harmed": 620, + "missed_rescue": 515, + "avoided_harm": 157, + "selected_both_correct": 7571, + "selected_both_wrong": 222 + }, + { + "budget": 0.78, + "threshold": -0.0299835205078125, + "operator": "ge", + "actual_fraction": 0.7733022636484687, + "metric": 0.22757452798038888, + "rescued": 798, + "harmed": 628, + "missed_rescue": 484, + "avoided_harm": 149, + "selected_both_correct": 7636, + "selected_both_wrong": 230 + }, + { + "budget": 0.79, + "threshold": -0.033966064453125, + "operator": "ge", + "actual_fraction": 0.784953395472703, + "metric": 0.227821050912831, + "rescued": 824, + "harmed": 634, + "missed_rescue": 458, + "avoided_harm": 143, + "selected_both_correct": 7728, + "selected_both_wrong": 246 + }, + { + "budget": 0.8, + "threshold": -0.037200927734375, + "operator": "ge", + "actual_fraction": 0.7939414114513982, + "metric": 0.22991865671924006, + "rescued": 852, + "harmed": 642, + "missed_rescue": 430, + "avoided_harm": 135, + "selected_both_correct": 7793, + "selected_both_wrong": 253 + }, + { + "budget": 0.81, + "threshold": -0.04132080078125, + "operator": "ge", + "actual_fraction": 0.8039280958721704, + "metric": 0.23486191981046528, + "rescued": 884, + "harmed": 646, + "missed_rescue": 398, + "avoided_harm": 131, + "selected_both_correct": 7868, + "selected_both_wrong": 262 + }, + { + "budget": 0.82, + "threshold": -0.0455322265625, + "operator": "ge", + "actual_fraction": 0.8122503328894807, + "metric": 0.23803916047544685, + "rescued": 906, + "harmed": 655, + "missed_rescue": 376, + "avoided_harm": 122, + "selected_both_correct": 7924, + "selected_both_wrong": 275 + }, + { + "budget": 0.83, + "threshold": -0.051849365234375, + "operator": "ge", + "actual_fraction": 0.8215712383488681, + "metric": 0.24022947987632814, + "rescued": 933, + "harmed": 664, + "missed_rescue": 349, + "avoided_harm": 113, + "selected_both_correct": 7987, + "selected_both_wrong": 288 + }, + { + "budget": 0.84, + "threshold": -0.06878662109375, + "operator": "ge", + "actual_fraction": 0.8388814913448736, + "metric": 0.24528627405740036, + "rescued": 988, + "harmed": 672, + "missed_rescue": 294, + "avoided_harm": 105, + "selected_both_correct": 8116, + "selected_both_wrong": 304 + }, + { + "budget": 0.85, + "threshold": -0.08477783203125, + "operator": "ge", + "actual_fraction": 0.8535286284953395, + "metric": 0.249537136326177, + "rescued": 1028, + "harmed": 680, + "missed_rescue": 254, + "avoided_harm": 97, + "selected_both_correct": 8224, + "selected_both_wrong": 324 + }, + { + "budget": 0.86, + "threshold": -0.09912109375, + "operator": "ge", + "actual_fraction": 0.8638482023968043, + "metric": 0.25211074301800224, + "rescued": 1056, + "harmed": 688, + "missed_rescue": 226, + "avoided_harm": 89, + "selected_both_correct": 8299, + "selected_both_wrong": 337 + }, + { + "budget": 0.87, + "threshold": -0.1298828125, + "operator": "ge", + "actual_fraction": 0.8745006657789614, + "metric": 0.2540042317315216, + "rescued": 1076, + "harmed": 695, + "missed_rescue": 206, + "avoided_harm": 82, + "selected_both_correct": 8389, + "selected_both_wrong": 348 + }, + { + "budget": 0.88, + "threshold": -0.1441650390625, + "operator": "ge", + "actual_fraction": 0.8764980026631158, + "metric": 0.2540996517958595, + "rescued": 1080, + "harmed": 698, + "missed_rescue": 202, + "avoided_harm": 79, + "selected_both_correct": 8404, + "selected_both_wrong": 350 + }, + { + "budget": 0.89, + "threshold": -0.182373046875, + "operator": "ge", + "actual_fraction": 0.8841544607190412, + "metric": 0.25436194678023316, + "rescued": 1090, + "harmed": 705, + "missed_rescue": 192, + "avoided_harm": 72, + "selected_both_correct": 8468, + "selected_both_wrong": 361 + }, + { + "budget": 0.9, + "threshold": -0.2049560546875, + "operator": "ge", + "actual_fraction": 0.8891478029294274, + "metric": 0.2534339447803836, + "rescued": 1101, + "harmed": 709, + "missed_rescue": 181, + "avoided_harm": 68, + "selected_both_correct": 8506, + "selected_both_wrong": 368 + }, + { + "budget": 0.91, + "threshold": -0.25830078125, + "operator": "gt", + "actual_fraction": 0.8981358189081226, + "metric": 0.2563240880787248, + "rescued": 1126, + "harmed": 715, + "missed_rescue": 156, + "avoided_harm": 62, + "selected_both_correct": 8573, + "selected_both_wrong": 378 + }, + { + "budget": 0.92, + "threshold": -0.32080078125, + "operator": "ge", + "actual_fraction": 0.9054593874833555, + "metric": 0.25551845731687656, + "rescued": 1143, + "harmed": 721, + "missed_rescue": 139, + "avoided_harm": 56, + "selected_both_correct": 8632, + "selected_both_wrong": 384 + }, + { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9221038615179761, + "metric": 0.2597490004762105, + "rescued": 1180, + "harmed": 728, + "missed_rescue": 102, + "avoided_harm": 49, + "selected_both_correct": 8774, + "selected_both_wrong": 398 + }, + { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9254327563249002, + "metric": 0.26306323674307786, + "rescued": 1189, + "harmed": 729, + "missed_rescue": 93, + "avoided_harm": 48, + "selected_both_correct": 8800, + "selected_both_wrong": 402 + }, + { + "budget": 0.95, + "threshold": -0.611328125, + "operator": "ge", + "actual_fraction": 0.9364181091877497, + "metric": 0.26218185491449153, + "rescued": 1203, + "harmed": 738, + "missed_rescue": 79, + "avoided_harm": 39, + "selected_both_correct": 8897, + "selected_both_wrong": 414 + }, + { + "budget": 0.96, + "threshold": -0.70068359375, + "operator": "ge", + "actual_fraction": 0.953728362183755, + "metric": 0.26357052137555514, + "rescued": 1223, + "harmed": 754, + "missed_rescue": 59, + "avoided_harm": 23, + "selected_both_correct": 9055, + "selected_both_wrong": 428 + }, + { + "budget": 0.97, + "threshold": -0.75048828125, + "operator": "ge", + "actual_fraction": 0.96271637816245, + "metric": 0.26400290307863034, + "rescued": 1231, + "harmed": 758, + "missed_rescue": 51, + "avoided_harm": 19, + "selected_both_correct": 9150, + "selected_both_wrong": 429 + }, + { + "budget": 0.98, + "threshold": -0.7841796875, + "operator": "ge", + "actual_fraction": 0.9700399467376831, + "metric": 0.2647475724296899, + "rescued": 1237, + "harmed": 760, + "missed_rescue": 45, + "avoided_harm": 17, + "selected_both_correct": 9227, + "selected_both_wrong": 432 + }, + { + "budget": 0.99, + "threshold": -0.85107421875, + "operator": "ge", + "actual_fraction": 0.987017310252996, + "metric": 0.2649306125590882, + "rescued": 1257, + "harmed": 772, + "missed_rescue": 25, + "avoided_harm": 5, + "selected_both_correct": 9387, + "selected_both_wrong": 444 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.27001009067886245, + "rescued": 1282, + "harmed": 777, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9508, + "selected_both_wrong": 449 + } + ] + }, + "learned": { + "selected_budget_index": 50, + "validation_area_0_25": 0.23466194499464404, + "validation_point": { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.33126660689663334, + "rescued": 304, + "harmed": 95, + "missed_rescue": 3, + "avoided_harm": 125, + "selected_both_correct": 916, + "selected_both_wrong": 85 + }, + "test_point": { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4876830892143808, + "metric": 0.29875800068449215, + "rescued": 1249, + "harmed": 337, + "missed_rescue": 33, + "avoided_harm": 440, + "selected_both_correct": 3843, + "selected_both_wrong": 431 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.3274500370025635, + "operator": "ge", + "actual_fraction": 0.009985734664764621, + "metric": 0.18570977475658457, + "rescued": 12, + "harmed": 1, + "missed_rescue": 295, + "avoided_harm": 219, + "selected_both_correct": 12, + "selected_both_wrong": 3 + }, + { + "budget": 0.02, + "threshold": 2.223353862762451, + "operator": "ge", + "actual_fraction": 0.019971469329529243, + "metric": 0.19072678038582708, + "rescued": 24, + "harmed": 1, + "missed_rescue": 283, + "avoided_harm": 219, + "selected_both_correct": 26, + "selected_both_wrong": 5 + }, + { + "budget": 0.03, + "threshold": 2.105621099472046, + "operator": "ge", + "actual_fraction": 0.029957203994293864, + "metric": 0.1947361119928147, + "rescued": 33, + "harmed": 1, + "missed_rescue": 274, + "avoided_harm": 219, + "selected_both_correct": 40, + "selected_both_wrong": 10 + }, + { + "budget": 0.04, + "threshold": 1.9981958866119385, + "operator": "ge", + "actual_fraction": 0.039942938659058486, + "metric": 0.20027934358652552, + "rescued": 44, + "harmed": 1, + "missed_rescue": 263, + "avoided_harm": 219, + "selected_both_correct": 52, + "selected_both_wrong": 15 + }, + { + "budget": 0.05, + "threshold": 1.949358582496643, + "operator": "ge", + "actual_fraction": 0.04992867332382311, + "metric": 0.21085585634765963, + "rescued": 57, + "harmed": 1, + "missed_rescue": 250, + "avoided_harm": 219, + "selected_both_correct": 65, + "selected_both_wrong": 17 + }, + { + "budget": 0.06, + "threshold": 1.8504760265350342, + "operator": "ge", + "actual_fraction": 0.05991440798858773, + "metric": 0.2155935788086818, + "rescued": 68, + "harmed": 2, + "missed_rescue": 239, + "avoided_harm": 218, + "selected_both_correct": 78, + "selected_both_wrong": 20 + }, + { + "budget": 0.07, + "threshold": 1.6561405658721924, + "operator": "ge", + "actual_fraction": 0.06990014265335236, + "metric": 0.2201008442387753, + "rescued": 76, + "harmed": 4, + "missed_rescue": 231, + "avoided_harm": 216, + "selected_both_correct": 93, + "selected_both_wrong": 23 + }, + { + "budget": 0.08, + "threshold": 1.5329420566558838, + "operator": "ge", + "actual_fraction": 0.07988587731811697, + "metric": 0.22831044506816361, + "rescued": 90, + "harmed": 5, + "missed_rescue": 217, + "avoided_harm": 215, + "selected_both_correct": 106, + "selected_both_wrong": 23 + }, + { + "budget": 0.09, + "threshold": 1.380273461341858, + "operator": "ge", + "actual_fraction": 0.0898716119828816, + "metric": 0.23143136333139114, + "rescued": 96, + "harmed": 6, + "missed_rescue": 211, + "avoided_harm": 214, + "selected_both_correct": 124, + "selected_both_wrong": 26 + }, + { + "budget": 0.1, + "threshold": 1.32539701461792, + "operator": "ge", + "actual_fraction": 0.09985734664764621, + "metric": 0.23760484070252644, + "rescued": 104, + "harmed": 8, + "missed_rescue": 203, + "avoided_harm": 212, + "selected_both_correct": 140, + "selected_both_wrong": 28 + }, + { + "budget": 0.11, + "threshold": 1.2173664569854736, + "operator": "ge", + "actual_fraction": 0.10984308131241084, + "metric": 0.23824320355020984, + "rescued": 110, + "harmed": 9, + "missed_rescue": 197, + "avoided_harm": 211, + "selected_both_correct": 161, + "selected_both_wrong": 28 + }, + { + "budget": 0.12, + "threshold": 1.1719112396240234, + "operator": "ge", + "actual_fraction": 0.11982881597717546, + "metric": 0.2407100587309389, + "rescued": 115, + "harmed": 9, + "missed_rescue": 192, + "avoided_harm": 211, + "selected_both_correct": 181, + "selected_both_wrong": 31 + }, + { + "budget": 0.13, + "threshold": 1.1186922788619995, + "operator": "ge", + "actual_fraction": 0.12981455064194009, + "metric": 0.24134727061556333, + "rescued": 121, + "harmed": 11, + "missed_rescue": 186, + "avoided_harm": 209, + "selected_both_correct": 199, + "selected_both_wrong": 33 + }, + { + "budget": 0.14, + "threshold": 1.0748379230499268, + "operator": "ge", + "actual_fraction": 0.1398002853067047, + "metric": 0.24423106111883697, + "rescued": 128, + "harmed": 16, + "missed_rescue": 179, + "avoided_harm": 204, + "selected_both_correct": 213, + "selected_both_wrong": 35 + }, + { + "budget": 0.15, + "threshold": 1.0427184104919434, + "operator": "ge", + "actual_fraction": 0.14978601997146934, + "metric": 0.24892068530842207, + "rescued": 135, + "harmed": 16, + "missed_rescue": 172, + "avoided_harm": 204, + "selected_both_correct": 234, + "selected_both_wrong": 35 + }, + { + "budget": 0.16, + "threshold": 1.0111123323440552, + "operator": "ge", + "actual_fraction": 0.15977175463623394, + "metric": 0.252253667728857, + "rescued": 141, + "harmed": 16, + "missed_rescue": 166, + "avoided_harm": 204, + "selected_both_correct": 254, + "selected_both_wrong": 37 + }, + { + "budget": 0.17, + "threshold": 0.9669641852378845, + "operator": "ge", + "actual_fraction": 0.16975748930099857, + "metric": 0.25275417712816833, + "rescued": 146, + "harmed": 18, + "missed_rescue": 161, + "avoided_harm": 202, + "selected_both_correct": 272, + "selected_both_wrong": 40 + }, + { + "budget": 0.18, + "threshold": 0.9490972757339478, + "operator": "ge", + "actual_fraction": 0.1797432239657632, + "metric": 0.25643535926629674, + "rescued": 152, + "harmed": 19, + "missed_rescue": 155, + "avoided_harm": 201, + "selected_both_correct": 292, + "selected_both_wrong": 41 + }, + { + "budget": 0.19, + "threshold": 0.9165292382240295, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.2576337529504327, + "rescued": 159, + "harmed": 22, + "missed_rescue": 148, + "avoided_harm": 198, + "selected_both_correct": 310, + "selected_both_wrong": 41 + }, + { + "budget": 0.2, + "threshold": 0.8978593349456787, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.2581122448979592, + "rescued": 164, + "harmed": 24, + "missed_rescue": 143, + "avoided_harm": 196, + "selected_both_correct": 330, + "selected_both_wrong": 42 + }, + { + "budget": 0.21, + "threshold": 0.854847252368927, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.2602060885029815, + "rescued": 169, + "harmed": 25, + "missed_rescue": 138, + "avoided_harm": 195, + "selected_both_correct": 350, + "selected_both_wrong": 44 + }, + { + "budget": 0.22, + "threshold": 0.8254434466362, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.2635325636000918, + "rescued": 174, + "harmed": 25, + "missed_rescue": 133, + "avoided_harm": 195, + "selected_both_correct": 371, + "selected_both_wrong": 46 + }, + { + "budget": 0.23, + "threshold": 0.8046320080757141, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.24799578059071728, + "rescued": 178, + "harmed": 29, + "missed_rescue": 129, + "avoided_harm": 191, + "selected_both_correct": 391, + "selected_both_wrong": 46 + }, + { + "budget": 0.24, + "threshold": 0.7761231660842896, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.25061334511913563, + "rescued": 184, + "harmed": 31, + "missed_rescue": 123, + "avoided_harm": 189, + "selected_both_correct": 410, + "selected_both_wrong": 47 + }, + { + "budget": 0.25, + "threshold": 0.7452236413955688, + "operator": "ge", + "actual_fraction": 0.24964336661911554, + "metric": 0.25309147374364765, + "rescued": 190, + "harmed": 32, + "missed_rescue": 117, + "avoided_harm": 188, + "selected_both_correct": 430, + "selected_both_wrong": 48 + }, + { + "budget": 0.26, + "threshold": 0.7167489528656006, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.25499133574730426, + "rescued": 196, + "harmed": 34, + "missed_rescue": 111, + "avoided_harm": 186, + "selected_both_correct": 448, + "selected_both_wrong": 50 + }, + { + "budget": 0.27, + "threshold": 0.6837625503540039, + "operator": "ge", + "actual_fraction": 0.26961483594864477, + "metric": 0.25870776744375423, + "rescued": 201, + "harmed": 34, + "missed_rescue": 106, + "avoided_harm": 186, + "selected_both_correct": 468, + "selected_both_wrong": 53 + }, + { + "budget": 0.28, + "threshold": 0.633089005947113, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.25965649053463696, + "rescued": 205, + "harmed": 36, + "missed_rescue": 102, + "avoided_harm": 184, + "selected_both_correct": 488, + "selected_both_wrong": 55 + }, + { + "budget": 0.29, + "threshold": 0.5875619649887085, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.2629594041536389, + "rescued": 213, + "harmed": 39, + "missed_rescue": 94, + "avoided_harm": 181, + "selected_both_correct": 505, + "selected_both_wrong": 55 + }, + { + "budget": 0.3, + "threshold": 0.5604347586631775, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.26353457364905875, + "rescued": 219, + "harmed": 45, + "missed_rescue": 88, + "avoided_harm": 175, + "selected_both_correct": 521, + "selected_both_wrong": 55 + }, + { + "budget": 0.31, + "threshold": 0.5320743918418884, + "operator": "ge", + "actual_fraction": 0.3095577746077033, + "metric": 0.26608946608946615, + "rescued": 225, + "harmed": 47, + "missed_rescue": 82, + "avoided_harm": 173, + "selected_both_correct": 540, + "selected_both_wrong": 56 + }, + { + "budget": 0.32, + "threshold": 0.49417412281036377, + "operator": "ge", + "actual_fraction": 0.3195435092724679, + "metric": 0.2673115302724549, + "rescued": 230, + "harmed": 51, + "missed_rescue": 77, + "avoided_harm": 169, + "selected_both_correct": 558, + "selected_both_wrong": 57 + }, + { + "budget": 0.33, + "threshold": 0.46743881702423096, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.26933684672815106, + "rescued": 236, + "harmed": 53, + "missed_rescue": 71, + "avoided_harm": 167, + "selected_both_correct": 577, + "selected_both_wrong": 58 + }, + { + "budget": 0.34, + "threshold": 0.4429023861885071, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.27352180339985216, + "rescued": 243, + "harmed": 54, + "missed_rescue": 64, + "avoided_harm": 166, + "selected_both_correct": 597, + "selected_both_wrong": 58 + }, + { + "budget": 0.35, + "threshold": 0.40694165229797363, + "operator": "ge", + "actual_fraction": 0.34950071326676174, + "metric": 0.275880713431076, + "rescued": 247, + "harmed": 60, + "missed_rescue": 60, + "avoided_harm": 160, + "selected_both_correct": 612, + "selected_both_wrong": 61 + }, + { + "budget": 0.36, + "threshold": 0.3611782193183899, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.27761867831050957, + "rescued": 253, + "harmed": 64, + "missed_rescue": 54, + "avoided_harm": 156, + "selected_both_correct": 629, + "selected_both_wrong": 62 + }, + { + "budget": 0.37, + "threshold": 0.3311936855316162, + "operator": "ge", + "actual_fraction": 0.369472182596291, + "metric": 0.28017336393459635, + "rescued": 257, + "harmed": 66, + "missed_rescue": 50, + "avoided_harm": 154, + "selected_both_correct": 650, + "selected_both_wrong": 63 + }, + { + "budget": 0.38, + "threshold": 0.28899651765823364, + "operator": "ge", + "actual_fraction": 0.37945791726105566, + "metric": 0.2825320512820513, + "rescued": 263, + "harmed": 70, + "missed_rescue": 44, + "avoided_harm": 150, + "selected_both_correct": 667, + "selected_both_wrong": 64 + }, + { + "budget": 0.39, + "threshold": 0.19986218214035034, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.28580355455355455, + "rescued": 267, + "harmed": 71, + "missed_rescue": 40, + "avoided_harm": 149, + "selected_both_correct": 689, + "selected_both_wrong": 65 + }, + { + "budget": 0.4, + "threshold": 0.157886803150177, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.3015271169266015, + "rescued": 271, + "harmed": 72, + "missed_rescue": 36, + "avoided_harm": 148, + "selected_both_correct": 709, + "selected_both_wrong": 68 + }, + { + "budget": 0.41, + "threshold": 0.12043046951293945, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.30656010656010657, + "rescued": 275, + "harmed": 74, + "missed_rescue": 32, + "avoided_harm": 146, + "selected_both_correct": 730, + "selected_both_wrong": 69 + }, + { + "budget": 0.42, + "threshold": 0.06623327732086182, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.30573593073593075, + "rescued": 277, + "harmed": 77, + "missed_rescue": 30, + "avoided_harm": 143, + "selected_both_correct": 750, + "selected_both_wrong": 72 + }, + { + "budget": 0.43, + "threshold": -0.01827472448348999, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.3122803645750104, + "rescued": 283, + "harmed": 79, + "missed_rescue": 24, + "avoided_harm": 141, + "selected_both_correct": 770, + "selected_both_wrong": 72 + }, + { + "budget": 0.44, + "threshold": -0.14177680015563965, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.32014301599541106, + "rescued": 290, + "harmed": 81, + "missed_rescue": 17, + "avoided_harm": 139, + "selected_both_correct": 788, + "selected_both_wrong": 73 + }, + { + "budget": 0.45, + "threshold": -0.270923376083374, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.3274352926559858, + "rescued": 295, + "harmed": 82, + "missed_rescue": 12, + "avoided_harm": 138, + "selected_both_correct": 808, + "selected_both_wrong": 75 + }, + { + "budget": 0.46, + "threshold": -0.4213377833366394, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.3269493177387914, + "rescued": 299, + "harmed": 85, + "missed_rescue": 8, + "avoided_harm": 135, + "selected_both_correct": 826, + "selected_both_wrong": 78 + }, + { + "budget": 0.47, + "threshold": -0.6602996587753296, + "operator": "ge", + "actual_fraction": 0.4693295292439372, + "metric": 0.3206497235233282, + "rescued": 299, + "harmed": 88, + "missed_rescue": 8, + "avoided_harm": 132, + "selected_both_correct": 849, + "selected_both_wrong": 80 + }, + { + "budget": 0.48, + "threshold": -0.8612939715385437, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.3201660892787428, + "rescued": 300, + "harmed": 89, + "missed_rescue": 7, + "avoided_harm": 131, + "selected_both_correct": 873, + "selected_both_wrong": 82 + }, + { + "budget": 0.49, + "threshold": -0.9715782999992371, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.325869430652523, + "rescued": 303, + "harmed": 91, + "missed_rescue": 4, + "avoided_harm": 129, + "selected_both_correct": 894, + "selected_both_wrong": 84 + }, + { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.33126660689663334, + "rescued": 304, + "harmed": 95, + "missed_rescue": 3, + "avoided_harm": 125, + "selected_both_correct": 916, + "selected_both_wrong": 85 + }, + { + "budget": 0.51, + "threshold": -1.6505446434020996, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.3264586577163139, + "rescued": 304, + "harmed": 98, + "missed_rescue": 3, + "avoided_harm": 122, + "selected_both_correct": 941, + "selected_both_wrong": 85 + }, + { + "budget": 0.52, + "threshold": -2.6140921115875244, + "operator": "ge", + "actual_fraction": 0.5192582025677603, + "metric": 0.3264586577163139, + "rescued": 304, + "harmed": 98, + "missed_rescue": 3, + "avoided_harm": 122, + "selected_both_correct": 968, + "selected_both_wrong": 86 + }, + { + "budget": 0.53, + "threshold": -2.6907107830047607, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.3235189795049722, + "rescued": 304, + "harmed": 102, + "missed_rescue": 3, + "avoided_harm": 118, + "selected_both_correct": 992, + "selected_both_wrong": 86 + }, + { + "budget": 0.54, + "threshold": -2.8344802856445312, + "operator": "ge", + "actual_fraction": 0.5392296718972895, + "metric": 0.3235189795049722, + "rescued": 304, + "harmed": 102, + "missed_rescue": 3, + "avoided_harm": 118, + "selected_both_correct": 1020, + "selected_both_wrong": 86 + }, + { + "budget": 0.55, + "threshold": -2.905412197113037, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.3203578942095772, + "rescued": 304, + "harmed": 106, + "missed_rescue": 3, + "avoided_harm": 114, + "selected_both_correct": 1044, + "selected_both_wrong": 86 + }, + { + "budget": 0.56, + "threshold": -2.9482638835906982, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.31783773291925466, + "rescued": 304, + "harmed": 107, + "missed_rescue": 3, + "avoided_harm": 113, + "selected_both_correct": 1071, + "selected_both_wrong": 86 + }, + { + "budget": 0.57, + "threshold": -2.9957408905029297, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.31989933342152455, + "rescued": 305, + "harmed": 111, + "missed_rescue": 2, + "avoided_harm": 109, + "selected_both_correct": 1094, + "selected_both_wrong": 86 + }, + { + "budget": 0.58, + "threshold": -3.1037726402282715, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.31989933342152455, + "rescued": 305, + "harmed": 111, + "missed_rescue": 2, + "avoided_harm": 109, + "selected_both_correct": 1122, + "selected_both_wrong": 86 + }, + { + "budget": 0.59, + "threshold": -3.1757895946502686, + "operator": "ge", + "actual_fraction": 0.5891583452211127, + "metric": 0.3184090005781182, + "rescued": 305, + "harmed": 113, + "missed_rescue": 2, + "avoided_harm": 107, + "selected_both_correct": 1148, + "selected_both_wrong": 86 + }, + { + "budget": 0.6, + "threshold": -3.253206729888916, + "operator": "ge", + "actual_fraction": 0.5991440798858774, + "metric": 0.3177705940408353, + "rescued": 305, + "harmed": 114, + "missed_rescue": 2, + "avoided_harm": 106, + "selected_both_correct": 1175, + "selected_both_wrong": 86 + }, + { + "budget": 0.61, + "threshold": -3.2825026512145996, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.31694234643839414, + "rescued": 305, + "harmed": 115, + "missed_rescue": 2, + "avoided_harm": 105, + "selected_both_correct": 1202, + "selected_both_wrong": 86 + }, + { + "budget": 0.62, + "threshold": -3.322240114212036, + "operator": "ge", + "actual_fraction": 0.6191155492154066, + "metric": 0.3111129112312657, + "rescued": 305, + "harmed": 121, + "missed_rescue": 2, + "avoided_harm": 99, + "selected_both_correct": 1224, + "selected_both_wrong": 86 + }, + { + "budget": 0.63, + "threshold": -3.3503003120422363, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.30995657450601133, + "rescued": 305, + "harmed": 123, + "missed_rescue": 2, + "avoided_harm": 97, + "selected_both_correct": 1250, + "selected_both_wrong": 86 + }, + { + "budget": 0.64, + "threshold": -3.377089738845825, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.3083114768968865, + "rescued": 305, + "harmed": 126, + "missed_rescue": 2, + "avoided_harm": 94, + "selected_both_correct": 1275, + "selected_both_wrong": 86 + }, + { + "budget": 0.65, + "threshold": -3.4055864810943604, + "operator": "ge", + "actual_fraction": 0.6490727532097005, + "metric": 0.30666904687523244, + "rescued": 305, + "harmed": 128, + "missed_rescue": 2, + "avoided_harm": 92, + "selected_both_correct": 1301, + "selected_both_wrong": 86 + }, + { + "budget": 0.66, + "threshold": -3.430248498916626, + "operator": "ge", + "actual_fraction": 0.6590584878744651, + "metric": 0.30339954163483573, + "rescued": 305, + "harmed": 131, + "missed_rescue": 2, + "avoided_harm": 89, + "selected_both_correct": 1326, + "selected_both_wrong": 86 + }, + { + "budget": 0.67, + "threshold": -3.4584414958953857, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.3011204481792717, + "rescued": 305, + "harmed": 135, + "missed_rescue": 2, + "avoided_harm": 85, + "selected_both_correct": 1350, + "selected_both_wrong": 86 + }, + { + "budget": 0.68, + "threshold": -3.502922773361206, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.2968337073049115, + "rescued": 305, + "harmed": 140, + "missed_rescue": 2, + "avoided_harm": 80, + "selected_both_correct": 1373, + "selected_both_wrong": 86 + }, + { + "budget": 0.69, + "threshold": -3.532865285873413, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.2968337073049115, + "rescued": 305, + "harmed": 140, + "missed_rescue": 2, + "avoided_harm": 80, + "selected_both_correct": 1401, + "selected_both_wrong": 86 + }, + { + "budget": 0.7, + "threshold": -3.5550942420959473, + "operator": "ge", + "actual_fraction": 0.6990014265335235, + "metric": 0.2951499808548299, + "rescued": 305, + "harmed": 143, + "missed_rescue": 2, + "avoided_harm": 77, + "selected_both_correct": 1426, + "selected_both_wrong": 86 + }, + { + "budget": 0.71, + "threshold": -3.566344976425171, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.2985697751322751, + "rescued": 306, + "harmed": 144, + "missed_rescue": 1, + "avoided_harm": 76, + "selected_both_correct": 1452, + "selected_both_wrong": 86 + }, + { + "budget": 0.72, + "threshold": -3.6056067943573, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.29288998820248824, + "rescued": 306, + "harmed": 150, + "missed_rescue": 1, + "avoided_harm": 70, + "selected_both_correct": 1474, + "selected_both_wrong": 86 + }, + { + "budget": 0.73, + "threshold": -3.628530979156494, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.2916863054994102, + "rescued": 306, + "harmed": 156, + "missed_rescue": 1, + "avoided_harm": 64, + "selected_both_correct": 1496, + "selected_both_wrong": 86 + }, + { + "budget": 0.74, + "threshold": -3.6580705642700195, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.2908398260842505, + "rescued": 306, + "harmed": 158, + "missed_rescue": 1, + "avoided_harm": 62, + "selected_both_correct": 1521, + "selected_both_wrong": 87 + }, + { + "budget": 0.75, + "threshold": -3.6900978088378906, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.29144523554686896, + "rescued": 307, + "harmed": 161, + "missed_rescue": 0, + "avoided_harm": 59, + "selected_both_correct": 1545, + "selected_both_wrong": 87 + }, + { + "budget": 0.76, + "threshold": -3.7050747871398926, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.28771367521367525, + "rescued": 307, + "harmed": 164, + "missed_rescue": 0, + "avoided_harm": 56, + "selected_both_correct": 1569, + "selected_both_wrong": 88 + }, + { + "budget": 0.77, + "threshold": -3.720116138458252, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.28342343829529665, + "rescued": 307, + "harmed": 170, + "missed_rescue": 0, + "avoided_harm": 50, + "selected_both_correct": 1591, + "selected_both_wrong": 88 + }, + { + "budget": 0.78, + "threshold": -3.7399919033050537, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.27967055835908294, + "rescued": 307, + "harmed": 175, + "missed_rescue": 0, + "avoided_harm": 45, + "selected_both_correct": 1614, + "selected_both_wrong": 88 + }, + { + "budget": 0.79, + "threshold": -3.799569606781006, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.2754483860214125, + "rescued": 307, + "harmed": 180, + "missed_rescue": 0, + "avoided_harm": 40, + "selected_both_correct": 1637, + "selected_both_wrong": 88 + }, + { + "budget": 0.8, + "threshold": -3.8980000019073486, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.2750612892472189, + "rescued": 307, + "harmed": 181, + "missed_rescue": 0, + "avoided_harm": 39, + "selected_both_correct": 1664, + "selected_both_wrong": 88 + }, + { + "budget": 0.81, + "threshold": -4.0607757568359375, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.27434520884520885, + "rescued": 307, + "harmed": 186, + "missed_rescue": 0, + "avoided_harm": 34, + "selected_both_correct": 1687, + "selected_both_wrong": 88 + }, + { + "budget": 0.82, + "threshold": -8.964031219482422, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.27434520884520885, + "rescued": 307, + "harmed": 186, + "missed_rescue": 0, + "avoided_harm": 34, + "selected_both_correct": 1715, + "selected_both_wrong": 88 + }, + { + "budget": 0.83, + "threshold": -9.02132511138916, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.27363625362132826, + "rescued": 307, + "harmed": 187, + "missed_rescue": 0, + "avoided_harm": 33, + "selected_both_correct": 1742, + "selected_both_wrong": 88 + }, + { + "budget": 0.84, + "threshold": -9.09000015258789, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.2722230802027571, + "rescued": 307, + "harmed": 189, + "missed_rescue": 0, + "avoided_harm": 31, + "selected_both_correct": 1768, + "selected_both_wrong": 88 + }, + { + "budget": 0.85, + "threshold": -9.12938404083252, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.2722230802027571, + "rescued": 307, + "harmed": 189, + "missed_rescue": 0, + "avoided_harm": 31, + "selected_both_correct": 1796, + "selected_both_wrong": 88 + }, + { + "budget": 0.86, + "threshold": -9.17154312133789, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.2708398536448072, + "rescued": 307, + "harmed": 191, + "missed_rescue": 0, + "avoided_harm": 29, + "selected_both_correct": 1822, + "selected_both_wrong": 88 + }, + { + "budget": 0.87, + "threshold": -9.273024559020996, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.2708398536448072, + "rescued": 307, + "harmed": 191, + "missed_rescue": 0, + "avoided_harm": 29, + "selected_both_correct": 1850, + "selected_both_wrong": 88 + }, + { + "budget": 0.88, + "threshold": -9.444879531860352, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.27016508981214865, + "rescued": 307, + "harmed": 192, + "missed_rescue": 0, + "avoided_harm": 28, + "selected_both_correct": 1877, + "selected_both_wrong": 88 + }, + { + "budget": 0.89, + "threshold": -9.566991806030273, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.27016508981214865, + "rescued": 307, + "harmed": 192, + "missed_rescue": 0, + "avoided_harm": 28, + "selected_both_correct": 1905, + "selected_both_wrong": 88 + }, + { + "budget": 0.9, + "threshold": -9.675031661987305, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.27016508981214865, + "rescued": 307, + "harmed": 192, + "missed_rescue": 0, + "avoided_harm": 28, + "selected_both_correct": 1933, + "selected_both_wrong": 88 + }, + { + "budget": 0.91, + "threshold": -9.739017486572266, + "operator": "ge", + "actual_fraction": 0.9087018544935807, + "metric": 0.2694835977031099, + "rescued": 307, + "harmed": 193, + "missed_rescue": 0, + "avoided_harm": 27, + "selected_both_correct": 1960, + "selected_both_wrong": 88 + }, + { + "budget": 0.92, + "threshold": -9.787779808044434, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.2680528165105032, + "rescued": 307, + "harmed": 196, + "missed_rescue": 0, + "avoided_harm": 24, + "selected_both_correct": 1985, + "selected_both_wrong": 88 + }, + { + "budget": 0.93, + "threshold": -9.832305908203125, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2680528165105032, + "rescued": 307, + "harmed": 196, + "missed_rescue": 0, + "avoided_harm": 24, + "selected_both_correct": 2013, + "selected_both_wrong": 88 + }, + { + "budget": 0.94, + "threshold": -9.88642692565918, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.2673844613676863, + "rescued": 307, + "harmed": 201, + "missed_rescue": 0, + "avoided_harm": 19, + "selected_both_correct": 2036, + "selected_both_wrong": 88 + }, + { + "budget": 0.95, + "threshold": -9.976344108581543, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.26575274061687104, + "rescued": 307, + "harmed": 203, + "missed_rescue": 0, + "avoided_harm": 17, + "selected_both_correct": 2062, + "selected_both_wrong": 88 + }, + { + "budget": 0.96, + "threshold": -10.03441047668457, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.261865308604439, + "rescued": 307, + "harmed": 209, + "missed_rescue": 0, + "avoided_harm": 11, + "selected_both_correct": 2084, + "selected_both_wrong": 88 + }, + { + "budget": 0.97, + "threshold": -10.085777282714844, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.2589514652014652, + "rescued": 307, + "harmed": 212, + "missed_rescue": 0, + "avoided_harm": 8, + "selected_both_correct": 2109, + "selected_both_wrong": 88 + }, + { + "budget": 0.98, + "threshold": -10.1488676071167, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.25830826953457287, + "rescued": 307, + "harmed": 213, + "missed_rescue": 0, + "avoided_harm": 7, + "selected_both_correct": 2136, + "selected_both_wrong": 88 + }, + { + "budget": 0.99, + "threshold": -10.265552520751953, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.25668767695229905, + "rescued": 307, + "harmed": 216, + "missed_rescue": 0, + "avoided_harm": 4, + "selected_both_correct": 2161, + "selected_both_wrong": 88 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.25448626222520154, + "rescued": 307, + "harmed": 220, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 2189, + "selected_both_wrong": 88 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.3274500370025635, + "operator": "ge", + "actual_fraction": 0.013648468708388815, + "metric": 0.18693471712069915, + "rescued": 72, + "harmed": 3, + "missed_rescue": 1210, + "avoided_harm": 774, + "selected_both_correct": 69, + "selected_both_wrong": 20 + }, + { + "budget": 0.02, + "threshold": 2.223353862762451, + "operator": "ge", + "actual_fraction": 0.019640479360852198, + "metric": 0.18781313491575913, + "rescued": 98, + "harmed": 7, + "missed_rescue": 1184, + "avoided_harm": 770, + "selected_both_correct": 104, + "selected_both_wrong": 27 + }, + { + "budget": 0.03, + "threshold": 2.105621099472046, + "operator": "ge", + "actual_fraction": 0.031291611185086554, + "metric": 0.1913128749335621, + "rescued": 154, + "harmed": 10, + "missed_rescue": 1128, + "avoided_harm": 767, + "selected_both_correct": 170, + "selected_both_wrong": 42 + }, + { + "budget": 0.04, + "threshold": 1.9981958866119385, + "operator": "ge", + "actual_fraction": 0.04061251664447404, + "metric": 0.19266008228682668, + "rescued": 193, + "harmed": 16, + "missed_rescue": 1089, + "avoided_harm": 761, + "selected_both_correct": 219, + "selected_both_wrong": 60 + }, + { + "budget": 0.05, + "threshold": 1.949358582496643, + "operator": "ge", + "actual_fraction": 0.045938748335552594, + "metric": 0.19531128485766117, + "rescued": 224, + "harmed": 18, + "missed_rescue": 1058, + "avoided_harm": 759, + "selected_both_correct": 249, + "selected_both_wrong": 61 + }, + { + "budget": 0.06, + "threshold": 1.8504760265350342, + "operator": "ge", + "actual_fraction": 0.0559254327563249, + "metric": 0.19800731707634547, + "rescued": 268, + "harmed": 23, + "missed_rescue": 1014, + "avoided_harm": 754, + "selected_both_correct": 302, + "selected_both_wrong": 79 + }, + { + "budget": 0.07, + "threshold": 1.6561405658721924, + "operator": "ge", + "actual_fraction": 0.07456724367509987, + "metric": 0.19763843353437874, + "rescued": 336, + "harmed": 37, + "missed_rescue": 946, + "avoided_harm": 740, + "selected_both_correct": 409, + "selected_both_wrong": 114 + }, + { + "budget": 0.08, + "threshold": 1.5329420566558838, + "operator": "ge", + "actual_fraction": 0.08688415446071904, + "metric": 0.1974112776278713, + "rescued": 386, + "harmed": 48, + "missed_rescue": 896, + "avoided_harm": 729, + "selected_both_correct": 472, + "selected_both_wrong": 138 + }, + { + "budget": 0.09, + "threshold": 1.380273461341858, + "operator": "ge", + "actual_fraction": 0.09587217043941411, + "metric": 0.20030220967092682, + "rescued": 421, + "harmed": 54, + "missed_rescue": 861, + "avoided_harm": 723, + "selected_both_correct": 529, + "selected_both_wrong": 148 + }, + { + "budget": 0.1, + "threshold": 1.32539701461792, + "operator": "ge", + "actual_fraction": 0.09920106524633822, + "metric": 0.20211074868345033, + "rescued": 432, + "harmed": 56, + "missed_rescue": 850, + "avoided_harm": 721, + "selected_both_correct": 552, + "selected_both_wrong": 152 + }, + { + "budget": 0.11, + "threshold": 1.2173664569854736, + "operator": "ge", + "actual_fraction": 0.11284953395472704, + "metric": 0.20834032699131655, + "rescued": 471, + "harmed": 62, + "missed_rescue": 811, + "avoided_harm": 715, + "selected_both_correct": 659, + "selected_both_wrong": 164 + }, + { + "budget": 0.12, + "threshold": 1.1719112396240234, + "operator": "ge", + "actual_fraction": 0.12350199733688415, + "metric": 0.21121674052765757, + "rescued": 503, + "harmed": 67, + "missed_rescue": 779, + "avoided_harm": 710, + "selected_both_correct": 745, + "selected_both_wrong": 169 + }, + { + "budget": 0.13, + "threshold": 1.1186922788619995, + "operator": "ge", + "actual_fraction": 0.13315579227696406, + "metric": 0.21276215593410724, + "rescued": 531, + "harmed": 77, + "missed_rescue": 751, + "avoided_harm": 700, + "selected_both_correct": 818, + "selected_both_wrong": 174 + }, + { + "budget": 0.14, + "threshold": 1.0748379230499268, + "operator": "ge", + "actual_fraction": 0.14347536617842876, + "metric": 0.21511521078428622, + "rescued": 556, + "harmed": 79, + "missed_rescue": 726, + "avoided_harm": 698, + "selected_both_correct": 907, + "selected_both_wrong": 182 + }, + { + "budget": 0.15, + "threshold": 1.0427184104919434, + "operator": "ge", + "actual_fraction": 0.15246338215712382, + "metric": 0.21738179676765032, + "rescued": 577, + "harmed": 81, + "missed_rescue": 705, + "avoided_harm": 696, + "selected_both_correct": 988, + "selected_both_wrong": 186 + }, + { + "budget": 0.16, + "threshold": 1.0111123323440552, + "operator": "ge", + "actual_fraction": 0.1604527296937417, + "metric": 0.22260454951009195, + "rescued": 597, + "harmed": 83, + "missed_rescue": 685, + "avoided_harm": 694, + "selected_both_correct": 1055, + "selected_both_wrong": 193 + }, + { + "budget": 0.17, + "threshold": 0.9669641852378845, + "operator": "ge", + "actual_fraction": 0.1780958721704394, + "metric": 0.22627631882842034, + "rescued": 641, + "harmed": 97, + "missed_rescue": 641, + "avoided_harm": 680, + "selected_both_correct": 1196, + "selected_both_wrong": 206 + }, + { + "budget": 0.18, + "threshold": 0.9490972757339478, + "operator": "ge", + "actual_fraction": 0.18442077230359522, + "metric": 0.2270967395967396, + "rescued": 657, + "harmed": 103, + "missed_rescue": 625, + "avoided_harm": 674, + "selected_both_correct": 1246, + "selected_both_wrong": 210 + }, + { + "budget": 0.19, + "threshold": 0.9165292382240295, + "operator": "ge", + "actual_fraction": 0.2000665778961385, + "metric": 0.23397114982462924, + "rescued": 697, + "harmed": 110, + "missed_rescue": 585, + "avoided_harm": 667, + "selected_both_correct": 1378, + "selected_both_wrong": 219 + }, + { + "budget": 0.2, + "threshold": 0.8978593349456787, + "operator": "ge", + "actual_fraction": 0.20439414114513982, + "metric": 0.23512276426997616, + "rescued": 707, + "harmed": 112, + "missed_rescue": 575, + "avoided_harm": 665, + "selected_both_correct": 1413, + "selected_both_wrong": 224 + }, + { + "budget": 0.21, + "threshold": 0.854847252368927, + "operator": "ge", + "actual_fraction": 0.21937416777629826, + "metric": 0.24202469545559713, + "rescued": 747, + "harmed": 119, + "missed_rescue": 535, + "avoided_harm": 658, + "selected_both_correct": 1539, + "selected_both_wrong": 231 + }, + { + "budget": 0.22, + "threshold": 0.8254434466362, + "operator": "ge", + "actual_fraction": 0.22769640479360853, + "metric": 0.24353835097564847, + "rescued": 766, + "harmed": 124, + "missed_rescue": 516, + "avoided_harm": 653, + "selected_both_correct": 1611, + "selected_both_wrong": 235 + }, + { + "budget": 0.23, + "threshold": 0.8046320080757141, + "operator": "ge", + "actual_fraction": 0.23601864181091878, + "metric": 0.2454657390209296, + "rescued": 787, + "harmed": 129, + "missed_rescue": 495, + "avoided_harm": 648, + "selected_both_correct": 1680, + "selected_both_wrong": 240 + }, + { + "budget": 0.24, + "threshold": 0.7761231660842896, + "operator": "ge", + "actual_fraction": 0.24533954727030627, + "metric": 0.2483414219175777, + "rescued": 812, + "harmed": 133, + "missed_rescue": 470, + "avoided_harm": 644, + "selected_both_correct": 1760, + "selected_both_wrong": 243 + }, + { + "budget": 0.25, + "threshold": 0.7452236413955688, + "operator": "ge", + "actual_fraction": 0.25432756324900135, + "metric": 0.2504846177578749, + "rescued": 831, + "harmed": 135, + "missed_rescue": 451, + "avoided_harm": 642, + "selected_both_correct": 1839, + "selected_both_wrong": 251 + }, + { + "budget": 0.26, + "threshold": 0.7167489528656006, + "operator": "ge", + "actual_fraction": 0.2636484687083888, + "metric": 0.252142216376585, + "rescued": 855, + "harmed": 144, + "missed_rescue": 427, + "avoided_harm": 633, + "selected_both_correct": 1914, + "selected_both_wrong": 255 + }, + { + "budget": 0.27, + "threshold": 0.6837625503540039, + "operator": "ge", + "actual_fraction": 0.275965379494008, + "metric": 0.25276746625574176, + "rescued": 880, + "harmed": 155, + "missed_rescue": 402, + "avoided_harm": 622, + "selected_both_correct": 2019, + "selected_both_wrong": 262 + }, + { + "budget": 0.28, + "threshold": 0.633089005947113, + "operator": "ge", + "actual_fraction": 0.29427430093209056, + "metric": 0.25702642690899347, + "rescued": 927, + "harmed": 172, + "missed_rescue": 355, + "avoided_harm": 605, + "selected_both_correct": 2167, + "selected_both_wrong": 270 + }, + { + "budget": 0.29, + "threshold": 0.5875619649887085, + "operator": "ge", + "actual_fraction": 0.31091877496671105, + "metric": 0.26183035419940204, + "rescued": 969, + "harmed": 193, + "missed_rescue": 313, + "avoided_harm": 584, + "selected_both_correct": 2296, + "selected_both_wrong": 278 + }, + { + "budget": 0.3, + "threshold": 0.5604347586631775, + "operator": "ge", + "actual_fraction": 0.322237017310253, + "metric": 0.2672887365132105, + "rescued": 995, + "harmed": 204, + "missed_rescue": 287, + "avoided_harm": 573, + "selected_both_correct": 2389, + "selected_both_wrong": 284 + }, + { + "budget": 0.31, + "threshold": 0.5320743918418884, + "operator": "ge", + "actual_fraction": 0.33388814913448733, + "metric": 0.27039536303602457, + "rescued": 1020, + "harmed": 216, + "missed_rescue": 262, + "avoided_harm": 561, + "selected_both_correct": 2480, + "selected_both_wrong": 296 + }, + { + "budget": 0.32, + "threshold": 0.49417412281036377, + "operator": "ge", + "actual_fraction": 0.34354194407456723, + "metric": 0.2713201570387918, + "rescued": 1038, + "harmed": 227, + "missed_rescue": 244, + "avoided_harm": 550, + "selected_both_correct": 2562, + "selected_both_wrong": 301 + }, + { + "budget": 0.33, + "threshold": 0.46743881702423096, + "operator": "ge", + "actual_fraction": 0.3492010652463382, + "metric": 0.2722481828097158, + "rescued": 1046, + "harmed": 230, + "missed_rescue": 236, + "avoided_harm": 547, + "selected_both_correct": 2610, + "selected_both_wrong": 310 + }, + { + "budget": 0.34, + "threshold": 0.4429023861885071, + "operator": "ge", + "actual_fraction": 0.3545272969374168, + "metric": 0.274417471554269, + "rescued": 1061, + "harmed": 239, + "missed_rescue": 221, + "avoided_harm": 538, + "selected_both_correct": 2648, + "selected_both_wrong": 312 + }, + { + "budget": 0.35, + "threshold": 0.40694165229797363, + "operator": "ge", + "actual_fraction": 0.3648468708388815, + "metric": 0.2782223498666452, + "rescued": 1081, + "harmed": 252, + "missed_rescue": 201, + "avoided_harm": 525, + "selected_both_correct": 2730, + "selected_both_wrong": 321 + }, + { + "budget": 0.36, + "threshold": 0.3611782193183899, + "operator": "ge", + "actual_fraction": 0.37217043941411454, + "metric": 0.2795152621397537, + "rescued": 1094, + "harmed": 257, + "missed_rescue": 188, + "avoided_harm": 520, + "selected_both_correct": 2793, + "selected_both_wrong": 328 + }, + { + "budget": 0.37, + "threshold": 0.3311936855316162, + "operator": "ge", + "actual_fraction": 0.37683089214380827, + "metric": 0.28090947933574756, + "rescued": 1104, + "harmed": 261, + "missed_rescue": 178, + "avoided_harm": 516, + "selected_both_correct": 2831, + "selected_both_wrong": 332 + }, + { + "budget": 0.38, + "threshold": 0.28899651765823364, + "operator": "ge", + "actual_fraction": 0.3854860186418109, + "metric": 0.28295009794869297, + "rescued": 1121, + "harmed": 269, + "missed_rescue": 161, + "avoided_harm": 508, + "selected_both_correct": 2904, + "selected_both_wrong": 338 + }, + { + "budget": 0.39, + "threshold": 0.19986218214035034, + "operator": "ge", + "actual_fraction": 0.39980026631158455, + "metric": 0.28644294895435674, + "rescued": 1145, + "harmed": 278, + "missed_rescue": 137, + "avoided_harm": 499, + "selected_both_correct": 3027, + "selected_both_wrong": 354 + }, + { + "budget": 0.4, + "threshold": 0.157886803150177, + "operator": "ge", + "actual_fraction": 0.4054593874833555, + "metric": 0.2862200047733393, + "rescued": 1160, + "harmed": 284, + "missed_rescue": 122, + "avoided_harm": 493, + "selected_both_correct": 3072, + "selected_both_wrong": 356 + }, + { + "budget": 0.41, + "threshold": 0.12043046951293945, + "operator": "ge", + "actual_fraction": 0.4104527296937417, + "metric": 0.28760386086589435, + "rescued": 1168, + "harmed": 287, + "missed_rescue": 114, + "avoided_harm": 490, + "selected_both_correct": 3115, + "selected_both_wrong": 362 + }, + { + "budget": 0.42, + "threshold": 0.06623327732086182, + "operator": "ge", + "actual_fraction": 0.4181091877496671, + "metric": 0.2930744270093899, + "rescued": 1181, + "harmed": 293, + "missed_rescue": 101, + "avoided_harm": 484, + "selected_both_correct": 3180, + "selected_both_wrong": 370 + }, + { + "budget": 0.43, + "threshold": -0.01827472448348999, + "operator": "ge", + "actual_fraction": 0.42809587217043943, + "metric": 0.2961685540536353, + "rescued": 1200, + "harmed": 302, + "missed_rescue": 82, + "avoided_harm": 475, + "selected_both_correct": 3262, + "selected_both_wrong": 380 + }, + { + "budget": 0.44, + "threshold": -0.14177680015563965, + "operator": "ge", + "actual_fraction": 0.4370838881491345, + "metric": 0.2977058723546182, + "rescued": 1215, + "harmed": 309, + "missed_rescue": 67, + "avoided_harm": 468, + "selected_both_correct": 3338, + "selected_both_wrong": 390 + }, + { + "budget": 0.45, + "threshold": -0.270923376083374, + "operator": "ge", + "actual_fraction": 0.44673768308921435, + "metric": 0.2999402430635043, + "rescued": 1230, + "harmed": 313, + "missed_rescue": 52, + "avoided_harm": 464, + "selected_both_correct": 3422, + "selected_both_wrong": 403 + }, + { + "budget": 0.46, + "threshold": -0.4213377833366394, + "operator": "ge", + "actual_fraction": 0.45206391478029295, + "metric": 0.2993046120720782, + "rescued": 1235, + "harmed": 316, + "missed_rescue": 47, + "avoided_harm": 461, + "selected_both_correct": 3475, + "selected_both_wrong": 406 + }, + { + "budget": 0.47, + "threshold": -0.6602996587753296, + "operator": "ge", + "actual_fraction": 0.45872170439414117, + "metric": 0.2998310238962732, + "rescued": 1243, + "harmed": 321, + "missed_rescue": 39, + "avoided_harm": 456, + "selected_both_correct": 3536, + "selected_both_wrong": 412 + }, + { + "budget": 0.48, + "threshold": -0.8612939715385437, + "operator": "ge", + "actual_fraction": 0.4673768308921438, + "metric": 0.30393211265289466, + "rescued": 1247, + "harmed": 326, + "missed_rescue": 35, + "avoided_harm": 451, + "selected_both_correct": 3623, + "selected_both_wrong": 420 + }, + { + "budget": 0.49, + "threshold": -0.9715782999992371, + "operator": "ge", + "actual_fraction": 0.47503328894806923, + "metric": 0.30350001249166053, + "rescued": 1248, + "harmed": 328, + "missed_rescue": 34, + "avoided_harm": 449, + "selected_both_correct": 3710, + "selected_both_wrong": 422 + }, + { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4876830892143808, + "metric": 0.29875800068449215, + "rescued": 1249, + "harmed": 337, + "missed_rescue": 33, + "avoided_harm": 440, + "selected_both_correct": 3843, + "selected_both_wrong": 431 + }, + { + "budget": 0.51, + "threshold": -1.6505446434020996, + "operator": "ge", + "actual_fraction": 0.5046604527296937, + "metric": 0.29989697064322696, + "rescued": 1255, + "harmed": 348, + "missed_rescue": 27, + "avoided_harm": 429, + "selected_both_correct": 4026, + "selected_both_wrong": 435 + }, + { + "budget": 0.52, + "threshold": -2.6140921115875244, + "operator": "ge", + "actual_fraction": 0.5206391478029294, + "metric": 0.297424793956044, + "rescued": 1255, + "harmed": 359, + "missed_rescue": 27, + "avoided_harm": 418, + "selected_both_correct": 4205, + "selected_both_wrong": 437 + }, + { + "budget": 0.53, + "threshold": -2.6907107830047607, + "operator": "ge", + "actual_fraction": 0.5249667110519307, + "metric": 0.297424793956044, + "rescued": 1255, + "harmed": 359, + "missed_rescue": 27, + "avoided_harm": 418, + "selected_both_correct": 4257, + "selected_both_wrong": 437 + }, + { + "budget": 0.54, + "threshold": -2.8344802856445312, + "operator": "ge", + "actual_fraction": 0.5436085219707057, + "metric": 0.2978194775414123, + "rescued": 1257, + "harmed": 377, + "missed_rescue": 25, + "avoided_harm": 400, + "selected_both_correct": 4461, + "selected_both_wrong": 437 + }, + { + "budget": 0.55, + "threshold": -2.905412197113037, + "operator": "ge", + "actual_fraction": 0.5529294274300932, + "metric": 0.29630990060928936, + "rescued": 1257, + "harmed": 385, + "missed_rescue": 25, + "avoided_harm": 392, + "selected_both_correct": 4565, + "selected_both_wrong": 437 + }, + { + "budget": 0.56, + "threshold": -2.9482638835906982, + "operator": "ge", + "actual_fraction": 0.5612516644474035, + "metric": 0.29578344205458756, + "rescued": 1257, + "harmed": 388, + "missed_rescue": 25, + "avoided_harm": 389, + "selected_both_correct": 4660, + "selected_both_wrong": 439 + }, + { + "budget": 0.57, + "threshold": -2.9957408905029297, + "operator": "ge", + "actual_fraction": 0.5692410119840213, + "metric": 0.2950949302773625, + "rescued": 1257, + "harmed": 396, + "missed_rescue": 25, + "avoided_harm": 381, + "selected_both_correct": 4748, + "selected_both_wrong": 439 + }, + { + "budget": 0.58, + "threshold": -3.1037726402282715, + "operator": "ge", + "actual_fraction": 0.5902130492676432, + "metric": 0.2947373376338526, + "rescued": 1260, + "harmed": 407, + "missed_rescue": 22, + "avoided_harm": 370, + "selected_both_correct": 4985, + "selected_both_wrong": 440 + }, + { + "budget": 0.59, + "threshold": -3.1757895946502686, + "operator": "ge", + "actual_fraction": 0.6021970705725699, + "metric": 0.29722091277674884, + "rescued": 1261, + "harmed": 417, + "missed_rescue": 21, + "avoided_harm": 360, + "selected_both_correct": 5115, + "selected_both_wrong": 443 + }, + { + "budget": 0.6, + "threshold": -3.253206729888916, + "operator": "ge", + "actual_fraction": 0.6208388814913449, + "metric": 0.29665021956289706, + "rescued": 1264, + "harmed": 430, + "missed_rescue": 18, + "avoided_harm": 347, + "selected_both_correct": 5323, + "selected_both_wrong": 443 + }, + { + "budget": 0.61, + "threshold": -3.2825026512145996, + "operator": "ge", + "actual_fraction": 0.6288282290279628, + "metric": 0.295988506182091, + "rescued": 1264, + "harmed": 435, + "missed_rescue": 18, + "avoided_harm": 342, + "selected_both_correct": 5413, + "selected_both_wrong": 444 + }, + { + "budget": 0.62, + "threshold": -3.322240114212036, + "operator": "ge", + "actual_fraction": 0.6421438082556591, + "metric": 0.2939669175161438, + "rescued": 1264, + "harmed": 452, + "missed_rescue": 18, + "avoided_harm": 325, + "selected_both_correct": 5556, + "selected_both_wrong": 444 + }, + { + "budget": 0.63, + "threshold": -3.3503003120422363, + "operator": "ge", + "actual_fraction": 0.6491344873501997, + "metric": 0.29277362668991463, + "rescued": 1265, + "harmed": 463, + "missed_rescue": 17, + "avoided_harm": 314, + "selected_both_correct": 5628, + "selected_both_wrong": 444 + }, + { + "budget": 0.64, + "threshold": -3.377089738845825, + "operator": "ge", + "actual_fraction": 0.6571238348868176, + "metric": 0.2930821542583563, + "rescued": 1266, + "harmed": 468, + "missed_rescue": 16, + "avoided_harm": 309, + "selected_both_correct": 5718, + "selected_both_wrong": 444 + }, + { + "budget": 0.65, + "threshold": -3.4055864810943604, + "operator": "ge", + "actual_fraction": 0.6677762982689747, + "metric": 0.2892305220657417, + "rescued": 1266, + "harmed": 488, + "missed_rescue": 16, + "avoided_harm": 289, + "selected_both_correct": 5826, + "selected_both_wrong": 444 + }, + { + "budget": 0.66, + "threshold": -3.430248498916626, + "operator": "ge", + "actual_fraction": 0.6744340878828229, + "metric": 0.2915915088317972, + "rescued": 1267, + "harmed": 492, + "missed_rescue": 15, + "avoided_harm": 285, + "selected_both_correct": 5901, + "selected_both_wrong": 444 + }, + { + "budget": 0.67, + "threshold": -3.4584414958953857, + "operator": "ge", + "actual_fraction": 0.6854194407456724, + "metric": 0.2936540392680743, + "rescued": 1269, + "harmed": 501, + "missed_rescue": 13, + "avoided_harm": 276, + "selected_both_correct": 6022, + "selected_both_wrong": 444 + }, + { + "budget": 0.68, + "threshold": -3.502922773361206, + "operator": "ge", + "actual_fraction": 0.7047270306258322, + "metric": 0.29632332439424064, + "rescued": 1272, + "harmed": 523, + "missed_rescue": 10, + "avoided_harm": 254, + "selected_both_correct": 6229, + "selected_both_wrong": 444 + }, + { + "budget": 0.69, + "threshold": -3.532865285873413, + "operator": "ge", + "actual_fraction": 0.7157123834886817, + "metric": 0.2937246709263134, + "rescued": 1273, + "harmed": 540, + "missed_rescue": 9, + "avoided_harm": 237, + "selected_both_correct": 6342, + "selected_both_wrong": 445 + }, + { + "budget": 0.7, + "threshold": -3.5550942420959473, + "operator": "ge", + "actual_fraction": 0.7253661784287616, + "metric": 0.2952254059747369, + "rescued": 1274, + "harmed": 550, + "missed_rescue": 8, + "avoided_harm": 227, + "selected_both_correct": 6446, + "selected_both_wrong": 446 + }, + { + "budget": 0.71, + "threshold": -3.566344976425171, + "operator": "ge", + "actual_fraction": 0.7283621837549934, + "metric": 0.2945683782087344, + "rescued": 1274, + "harmed": 553, + "missed_rescue": 8, + "avoided_harm": 224, + "selected_both_correct": 6478, + "selected_both_wrong": 447 + }, + { + "budget": 0.72, + "threshold": -3.6056067943573, + "operator": "ge", + "actual_fraction": 0.741011984021305, + "metric": 0.29252963134197957, + "rescued": 1275, + "harmed": 568, + "missed_rescue": 7, + "avoided_harm": 209, + "selected_both_correct": 6614, + "selected_both_wrong": 447 + }, + { + "budget": 0.73, + "threshold": -3.628530979156494, + "operator": "ge", + "actual_fraction": 0.7526631158455392, + "metric": 0.2900395106945919, + "rescued": 1276, + "harmed": 588, + "missed_rescue": 6, + "avoided_harm": 189, + "selected_both_correct": 6733, + "selected_both_wrong": 447 + }, + { + "budget": 0.74, + "threshold": -3.6580705642700195, + "operator": "ge", + "actual_fraction": 0.7616511318242344, + "metric": 0.2887744568626224, + "rescued": 1276, + "harmed": 597, + "missed_rescue": 6, + "avoided_harm": 180, + "selected_both_correct": 6832, + "selected_both_wrong": 447 + }, + { + "budget": 0.75, + "threshold": -3.6900978088378906, + "operator": "ge", + "actual_fraction": 0.7709720372836218, + "metric": 0.28730364578560735, + "rescued": 1276, + "harmed": 606, + "missed_rescue": 6, + "avoided_harm": 171, + "selected_both_correct": 6935, + "selected_both_wrong": 447 + }, + { + "budget": 0.76, + "threshold": -3.7050747871398926, + "operator": "ge", + "actual_fraction": 0.7749667110519307, + "metric": 0.2858070539975931, + "rescued": 1276, + "harmed": 612, + "missed_rescue": 6, + "avoided_harm": 165, + "selected_both_correct": 6977, + "selected_both_wrong": 447 + }, + { + "budget": 0.77, + "threshold": -3.720116138458252, + "operator": "ge", + "actual_fraction": 0.7792942743009321, + "metric": 0.28488735368132817, + "rescued": 1276, + "harmed": 617, + "missed_rescue": 6, + "avoided_harm": 160, + "selected_both_correct": 7023, + "selected_both_wrong": 448 + }, + { + "budget": 0.78, + "threshold": -3.7399919033050537, + "operator": "ge", + "actual_fraction": 0.7846205059920106, + "metric": 0.2830658409372236, + "rescued": 1276, + "harmed": 627, + "missed_rescue": 6, + "avoided_harm": 150, + "selected_both_correct": 7077, + "selected_both_wrong": 448 + }, + { + "budget": 0.79, + "threshold": -3.799569606781006, + "operator": "ge", + "actual_fraction": 0.7972703062583223, + "metric": 0.2809995077380739, + "rescued": 1277, + "harmed": 643, + "missed_rescue": 5, + "avoided_harm": 134, + "selected_both_correct": 7212, + "selected_both_wrong": 448 + }, + { + "budget": 0.8, + "threshold": -3.8980000019073486, + "operator": "ge", + "actual_fraction": 0.8129161118508655, + "metric": 0.28013559922601305, + "rescued": 1278, + "harmed": 667, + "missed_rescue": 4, + "avoided_harm": 110, + "selected_both_correct": 7375, + "selected_both_wrong": 448 + }, + { + "budget": 0.81, + "threshold": -4.0607757568359375, + "operator": "ge", + "actual_fraction": 0.8199067909454061, + "metric": 0.27814841049220523, + "rescued": 1278, + "harmed": 681, + "missed_rescue": 4, + "avoided_harm": 96, + "selected_both_correct": 7445, + "selected_both_wrong": 448 + }, + { + "budget": 0.82, + "threshold": -8.964031219482422, + "operator": "ge", + "actual_fraction": 0.831225033288948, + "metric": 0.2772348763322643, + "rescued": 1278, + "harmed": 686, + "missed_rescue": 4, + "avoided_harm": 91, + "selected_both_correct": 7576, + "selected_both_wrong": 448 + }, + { + "budget": 0.83, + "threshold": -9.02132511138916, + "operator": "ge", + "actual_fraction": 0.84254327563249, + "metric": 0.27700595116508253, + "rescued": 1279, + "harmed": 689, + "missed_rescue": 3, + "avoided_harm": 88, + "selected_both_correct": 7708, + "selected_both_wrong": 448 + }, + { + "budget": 0.84, + "threshold": -9.09000015258789, + "operator": "ge", + "actual_fraction": 0.857190412782956, + "metric": 0.2763733409861609, + "rescued": 1279, + "harmed": 692, + "missed_rescue": 3, + "avoided_harm": 85, + "selected_both_correct": 7881, + "selected_both_wrong": 448 + }, + { + "budget": 0.85, + "threshold": -9.12938404083252, + "operator": "ge", + "actual_fraction": 0.8618508655126498, + "metric": 0.2761345157638265, + "rescued": 1279, + "harmed": 694, + "missed_rescue": 3, + "avoided_harm": 83, + "selected_both_correct": 7935, + "selected_both_wrong": 448 + }, + { + "budget": 0.86, + "threshold": -9.17154312133789, + "operator": "ge", + "actual_fraction": 0.87017310252996, + "metric": 0.2761345157638265, + "rescued": 1279, + "harmed": 694, + "missed_rescue": 3, + "avoided_harm": 83, + "selected_both_correct": 8035, + "selected_both_wrong": 448 + }, + { + "budget": 0.87, + "threshold": -9.273024559020996, + "operator": "ge", + "actual_fraction": 0.8848202396804261, + "metric": 0.27546832609774896, + "rescued": 1279, + "harmed": 699, + "missed_rescue": 3, + "avoided_harm": 78, + "selected_both_correct": 8206, + "selected_both_wrong": 448 + }, + { + "budget": 0.88, + "threshold": -9.444879531860352, + "operator": "ge", + "actual_fraction": 0.8984687083888149, + "metric": 0.27423981143437465, + "rescued": 1279, + "harmed": 709, + "missed_rescue": 3, + "avoided_harm": 68, + "selected_both_correct": 8360, + "selected_both_wrong": 448 + }, + { + "budget": 0.89, + "threshold": -9.566991806030273, + "operator": "ge", + "actual_fraction": 0.9087882822902796, + "metric": 0.2736575311942959, + "rescued": 1279, + "harmed": 713, + "missed_rescue": 3, + "avoided_harm": 64, + "selected_both_correct": 8480, + "selected_both_wrong": 448 + }, + { + "budget": 0.9, + "threshold": -9.675031661987305, + "operator": "ge", + "actual_fraction": 0.9161118508655126, + "metric": 0.272975172975173, + "rescued": 1279, + "harmed": 717, + "missed_rescue": 3, + "avoided_harm": 60, + "selected_both_correct": 8563, + "selected_both_wrong": 449 + }, + { + "budget": 0.91, + "threshold": -9.739017486572266, + "operator": "ge", + "actual_fraction": 0.9217709720372836, + "metric": 0.2727190231806067, + "rescued": 1279, + "harmed": 719, + "missed_rescue": 3, + "avoided_harm": 58, + "selected_both_correct": 8629, + "selected_both_wrong": 449 + }, + { + "budget": 0.92, + "threshold": -9.787779808044434, + "operator": "ge", + "actual_fraction": 0.9264314247669774, + "metric": 0.27264729817760425, + "rescued": 1279, + "harmed": 720, + "missed_rescue": 3, + "avoided_harm": 57, + "selected_both_correct": 8684, + "selected_both_wrong": 449 + }, + { + "budget": 0.93, + "threshold": -9.832305908203125, + "operator": "ge", + "actual_fraction": 0.933422103861518, + "metric": 0.2741441928343702, + "rescued": 1280, + "harmed": 724, + "missed_rescue": 2, + "avoided_harm": 53, + "selected_both_correct": 8763, + "selected_both_wrong": 449 + }, + { + "budget": 0.94, + "threshold": -9.88642692565918, + "operator": "ge", + "actual_fraction": 0.9417443408788282, + "metric": 0.2733716828250215, + "rescued": 1280, + "harmed": 728, + "missed_rescue": 2, + "avoided_harm": 49, + "selected_both_correct": 8859, + "selected_both_wrong": 449 + }, + { + "budget": 0.95, + "threshold": -9.976344108581543, + "operator": "ge", + "actual_fraction": 0.9530625832223701, + "metric": 0.2727312493845365, + "rescued": 1280, + "harmed": 737, + "missed_rescue": 2, + "avoided_harm": 40, + "selected_both_correct": 8986, + "selected_both_wrong": 449 + }, + { + "budget": 0.96, + "threshold": -10.03441047668457, + "operator": "ge", + "actual_fraction": 0.9613848202396804, + "metric": 0.27223682114986464, + "rescued": 1280, + "harmed": 740, + "missed_rescue": 2, + "avoided_harm": 37, + "selected_both_correct": 9083, + "selected_both_wrong": 449 + }, + { + "budget": 0.97, + "threshold": -10.085777282714844, + "operator": "ge", + "actual_fraction": 0.9673768308921438, + "metric": 0.27453093162048386, + "rescued": 1282, + "harmed": 744, + "missed_rescue": 0, + "avoided_harm": 33, + "selected_both_correct": 9149, + "selected_both_wrong": 449 + }, + { + "budget": 0.98, + "threshold": -10.1488676071167, + "operator": "ge", + "actual_fraction": 0.9760319573901465, + "metric": 0.2737331615154065, + "rescued": 1282, + "harmed": 750, + "missed_rescue": 0, + "avoided_harm": 27, + "selected_both_correct": 9247, + "selected_both_wrong": 449 + }, + { + "budget": 0.99, + "threshold": -10.265552520751953, + "operator": "ge", + "actual_fraction": 0.9890146471371505, + "metric": 0.2725383412883413, + "rescued": 1282, + "harmed": 758, + "missed_rescue": 0, + "avoided_harm": 19, + "selected_both_correct": 9395, + "selected_both_wrong": 449 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.27001009067886245, + "rescued": 1282, + "harmed": 777, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9508, + "selected_both_wrong": 449 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 2189, + "rescuable": 307, + "harmful": 220, + "both_wrong": 88, + "unit": "event", + "total": 2804 + }, + "test": { + "both_correct": 9508, + "rescuable": 1282, + "harmful": 777, + "both_wrong": 449, + "unit": "event", + "total": 12016 + } + }, + "small": { + "validation": 0.18254072168965788, + "test": 0.18129624718318377 + }, + "large": { + "validation": 0.25448626222520154, + "test": 0.27001009067886245 + }, + "learned_minus_confidence_test": 0.03900900020828163, + "learned_minus_small_test": 0.11746175350130839, + "learned_minus_large_test": 0.028747910005629707 + } + ] +} \ No newline at end of file diff --git a/paper_results/data/main_table_cost_qwen_20260912/reproducible/measure_flops.py b/paper_results/data/main_table_cost_qwen_20260912/reproducible/measure_flops.py new file mode 100644 index 0000000000000000000000000000000000000000..4b5eb79777bb9eca67245173f1c6813f000b8e8d --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/reproducible/measure_flops.py @@ -0,0 +1,152 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import argparse +import json +import re +from pathlib import Path + +import numpy as np +import torch +from PIL import Image +from torch.utils.flop_counter import FlopCounterMode + + +def count(call): + with torch.inference_mode(), FlopCounterMode(display=False) as counter: + value = call() + torch.cuda.synchronize() + counts = counter.get_flop_counts().get('Global', {}) + return int(counter.get_total_flops()), value, {str(k): int(v) for k, v in counts.items()} + + +def make_text(root: Path, model_kind: str, task: str, row: dict, processor): + prompt = (root / 'outputs/large_model_labels_all_20260909/prompts' / task / f'{model_kind}.txt').read_text(encoding='utf-8').strip() + if task == 'grefcoco': + prompt = prompt.replace('{expression}', row['expression']) + if model_kind == 'Qwen3.8': + content = [{'type': 'image'}, {'type': 'text', 'text': '\n'}, {'type': 'text', 'text': prompt}] + return processor.apply_chat_template([{'role': 'user', 'content': content}], tokenize=False, add_generation_prompt=True) + message = '(./)\n' + prompt + return processor.tokenizer.apply_chat_template([{'role': 'user', 'content': message}], tokenize=False, add_generation_prompt=True, enable_thinking=False) + + +def prepare(root: Path, model_kind: str, task: str, row: dict, processor): + image = Image.open(row['image_path']).convert('RGB') + text = make_text(root, model_kind, task, row, processor) + if model_kind == 'Qwen3.8': + data = processor(text=[text], images=[image], return_tensors='pt', size={'shortest_edge': 65536, 'longest_edge': 1003520}) + else: + data = processor([text], [[image]], return_tensors='pt', max_length=16384) + image.close() + return data + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('--root', type=Path, required=True) + parser.add_argument('--model-kind', choices=['Qwen3.8', 'MiniCPM'], required=True) + parser.add_argument('--task', choices=['grefcoco', 'construction'], required=True) + parser.add_argument('--output', type=Path, required=True) + parser.add_argument('--samples', type=int, default=4) + args = parser.parse_args() + if args.output.exists(): + raise FileExistsError(args.output) + expert = 'instancevg' if args.task == 'grefcoco' else 'yolo26x' + rows = [json.loads(line) for line in (args.root / f'outputs/router_full_20260910/data/{expert}/test/records.jsonl').open(encoding='utf-8')] + if args.samples > len(rows): + raise ValueError('sample count exceeds test rows') + selected_indices = np.random.default_rng(20260911).choice(len(rows), args.samples, replace=False) + selected = [rows[int(index)] for index in selected_indices] + checkpoint_root = args.root.parents[1] / 'checkpoints' + if not checkpoint_root.exists(): + checkpoint_root = args.root / 'checkpoints' + model_path = checkpoint_root / ('Qwen3.8-27B-FP8' if args.model_kind == 'Qwen3.8' else 'MiniCPM-V-4_5') + from transformers import AutoProcessor + processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True, local_files_only=True) + if args.model_kind == 'Qwen3.8': + from transformers import FineGrainedFP8Config, Qwen3_5ForConditionalGeneration + import transformers.models.qwen3_5.modeling_qwen3_5 as implementation + implementation.causal_conv1d_fn = None + implementation.causal_conv1d_update = None + implementation.chunk_gated_delta_rule = None + implementation.fused_recurrent_gated_delta_rule = None + implementation.FusedRMSNormGated = None + implementation.is_fast_path_available = False + model = Qwen3_5ForConditionalGeneration.from_pretrained( + model_path, local_files_only=True, dtype=torch.bfloat16, device_map={'': 'cuda:0'}, + attn_implementation='eager', quantization_config=FineGrainedFP8Config(dequantize=False), + ) + else: + model = processor # replaced below to keep imports local to the selected environment + from transformers import AutoModel + from transformers.modeling_utils import PreTrainedModel + if not hasattr(PreTrainedModel, 'all_tied_weights_keys'): + def get_tied_weights(self): + return getattr(self, '_all_tied_weights_keys', getattr(self, '_tied_weights_keys', {}) or {}) + def set_tied_weights(self, value): + self._all_tied_weights_keys = value + PreTrainedModel.all_tied_weights_keys = property(get_tied_weights, set_tied_weights) + model = AutoModel.from_pretrained( + model_path, trust_remote_code=True, local_files_only=True, torch_dtype=torch.bfloat16, + device_map={'': 'cuda:0'}, attn_implementation='eager', + ) + torch.set_num_threads(4) + torch.backends.mha.set_fastpath_enabled(False) + model.eval().requires_grad_(False) + records = [] + for row in selected: + data = prepare(args.root, args.model_kind, args.task, row, processor).to('cuda') + prompt_tokens = int(data['input_ids'].shape[-1]) + if args.model_kind == 'MiniCPM': + vision_flops, embedding_result, vision_ops = count(lambda: model.get_vllm_embedding(data)) + embeddings, _ = embedding_result + prefill_flops, prefill_output, prefill_ops = count(lambda: model.llm(inputs_embeds=embeddings, attention_mask=data['attention_mask'], use_cache=True, logits_to_keep=1)) + prefill_flops += vision_flops + prefill_ops.update({'vision:'+k:v for k,v in vision_ops.items()}) + length = int(data['input_ids'].shape[-1]) + known_token = data['input_ids'][:, -1:] + kwargs = { + 'input_ids': known_token, + 'past_key_values': prefill_output.past_key_values, + 'attention_mask': torch.ones((1, length + 1), device='cuda', dtype=torch.long), + 'use_cache': True, + 'logits_to_keep': 1, + } + decode_flops, _, decode_ops = count(lambda: model.llm(**kwargs)) + else: + prefill_flops, prefill_output, prefill_ops = count(lambda: model(**data, use_cache=True, logits_to_keep=1)) + length = int(data['input_ids'].shape[-1]) + known_token = data['input_ids'][:, -1:] + kwargs = { + 'input_ids': known_token, + 'past_key_values': prefill_output.past_key_values, + 'attention_mask': torch.ones((1, length + 1), device='cuda', dtype=torch.long), + 'use_cache': True, + 'logits_to_keep': 1, + 'cache_position': torch.tensor([length], device='cuda'), + } + kwargs.pop('attention_mask') + decode_flops, _, decode_ops = count(lambda: model(**kwargs)) + records.append({ + 'sample_id': row['sample_id'], 'prompt_tokens_in_reference_tensor': prompt_tokens, + 'input_shapes': {key: list(value.shape) for key, value in data.items() if hasattr(value, 'shape')}, + 'prefill_flops_counted': prefill_flops, 'decode_one_token_flops_counted': decode_flops, + 'prefill_gflops_counted': prefill_flops / 1e9, 'decode_one_token_gflops_counted': decode_flops / 1e9, + 'prefill_operators': prefill_ops, 'decode_operators': decode_ops, + }) + args.output.parent.mkdir(parents=True, exist_ok=True) + args.output.write_text(json.dumps({'status': 'running', 'model_kind': args.model_kind, 'task': args.task, 'records': records}, ensure_ascii=False, indent=2), encoding='utf-8') + print(json.dumps({'completed': len(records), 'sample_id': row['sample_id'], 'prefill_gflops': prefill_flops / 1e9, 'decode_gflops': decode_flops / 1e9}), flush=True) + args.output.write_text(json.dumps({ + 'status': 'complete', 'model_kind': args.model_kind, 'task': args.task, 'model_path': str(model_path), + 'sample_count': len(records), 'records': records, + 'mean_prefill_gflops_counted': float(np.mean([row['prefill_gflops_counted'] for row in records])), + 'mean_decode_one_token_gflops_counted': float(np.mean([row['decode_one_token_gflops_counted'] for row in records])), + 'scope': 'HF reference execution on the A800 with real task inputs; dense products/convolutions/supported attention counted with multiply-add=2 FLOPs. vLLM custom kernels, elementwise/normalization/nonlinearities and other unsupported operations are not complete.', + 'answer_generation': 'not performed; one known-token decode probe only', + }, ensure_ascii=False, indent=2), encoding='utf-8') + + +if __name__ == '__main__': + main() diff --git a/paper_results/data/main_table_cost_qwen_20260912/reproducible/measure_large.py b/paper_results/data/main_table_cost_qwen_20260912/reproducible/measure_large.py new file mode 100644 index 0000000000000000000000000000000000000000..61b54474c6d7e15ee107c15c3ca4eb10c14ebca3 --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/reproducible/measure_large.py @@ -0,0 +1,209 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import argparse +import json +import time +from pathlib import Path + +import numpy as np +import jsonschema +from PIL import Image +from vllm import LLM, SamplingParams +from vllm.sampling_params import StructuredOutputsParams + + +EVENTS = [ + 'rule_1_ppe_violation', + 'rule_2_fall_protection_violation', + 'rule_3_unprotected_edge_violation', + 'rule_4_excavator_proximity_violation', +] + + +def gref_schema(): + return { + 'type': 'object', + 'properties': { + 'objects': { + 'type': 'array', 'maxItems': 18, + 'items': { + 'type': 'object', + 'properties': { + 'bbox_2d': {'type': 'array', 'items': {'type': 'number', 'minimum': 0, 'maximum': 1000}, 'minItems': 4, 'maxItems': 4}, + 'score': {'type': 'number', 'minimum': 0, 'maximum': 1}, + }, + 'required': ['bbox_2d', 'score'], 'additionalProperties': False, + }, + }, + }, + 'required': ['objects'], 'additionalProperties': False, + } + + +def construction_schema(): + box = {'type': 'array', 'items': {'type': 'integer', 'minimum': 0, 'maximum': 1000}, 'minItems': 4, 'maxItems': 4} + max_boxes = [13, 6, 3, 2] + events = {} + for event, limit in zip(EVENTS, max_boxes): + events[event] = { + 'type': 'object', + 'properties': { + 'present': {'type': 'boolean'}, + 'reason': {'type': ['string', 'null']}, + 'boxes': {'type': 'array', 'items': box, 'maxItems': limit}, + }, + 'required': ['present', 'reason', 'boxes'], 'additionalProperties': False, + } + return {'type': 'object', 'properties': {'events': {'type': 'object', 'properties': events, 'required': EVENTS, 'additionalProperties': False}}, 'required': ['events'], 'additionalProperties': False} + + +def make_prompt(root: Path, model_kind: str, task: str, row: dict, processor_or_tokenizer): + prompt = (root / 'outputs/large_model_labels_all_20260909/prompts' / task / f'{model_kind}.txt').read_text(encoding='utf-8').strip() + if task == 'grefcoco': + prompt = prompt.replace('{expression}', row['expression']) + if model_kind == 'Qwen3.8': + converted = [{'type': 'image'}] + converted.append({'type': 'text', 'text': '\n'}) + converted.append({'type': 'text', 'text': prompt}) + messages = [{'role': 'user', 'content': converted}] + return processor_or_tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) + message = '(./)\n' + prompt + return processor_or_tokenizer.apply_chat_template([{'role': 'user', 'content': message}], tokenize=False, add_generation_prompt=True, enable_thinking=False) + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('--root', type=Path, required=True) + parser.add_argument('--model-kind', choices=['Qwen3.8', 'MiniCPM'], required=True) + parser.add_argument('--task', choices=['grefcoco', 'construction'], required=True) + parser.add_argument('--output', type=Path, required=True) + parser.add_argument('--samples', type=int, default=32) + parser.add_argument('--batch-size', type=int, default=8) + parser.add_argument('--gpu-memory-utilization', type=float, default=0.90) + parser.add_argument('--max-model-len', type=int, default=16384) + args = parser.parse_args() + if args.output.exists(): + raise FileExistsError(args.output) + if args.samples < 3 or args.batch_size < 1: + raise ValueError('samples must include warmup rows and batch size must be positive') + expert = 'instancevg' if args.task == 'grefcoco' else 'yolo26x' + rows = [json.loads(line) for line in (args.root / f'outputs/router_full_20260910/data/{expert}/test/records.jsonl').open(encoding='utf-8')] + if args.samples > len(rows): + raise ValueError(f'samples {args.samples} exceeds {len(rows)}') + selected_indices = np.random.default_rng(20260911).choice(len(rows), args.samples, replace=False) + selected = [rows[int(index)] for index in selected_indices] + checkpoint_root = args.root.parents[1] / 'checkpoints' + if not checkpoint_root.exists(): + checkpoint_root = args.root / 'checkpoints' + model_path = checkpoint_root / ('Qwen3.8-27B-FP8' if args.model_kind == 'Qwen3.8' else 'MiniCPM-V-4_5') + if args.model_kind == 'Qwen3.8': + from transformers import AutoProcessor + processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True, local_files_only=True) + model_kwargs = {'mm_processor_kwargs': {'size': {'shortest_edge': 65536, 'longest_edge': 1003520}}} + else: + from transformers import AutoTokenizer + processor = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True, local_files_only=True) + model_kwargs = {} + model = LLM(model=str(model_path), trust_remote_code=True, tensor_parallel_size=1, max_model_len=args.max_model_len, + limit_mm_per_prompt={'image': 1}, max_num_seqs=args.batch_size, + gpu_memory_utilization=args.gpu_memory_utilization, enforce_eager=True, **model_kwargs) + schema = gref_schema() if args.task == 'grefcoco' else construction_schema() + sampling = SamplingParams(temperature=0, max_tokens=512 if args.task == 'grefcoco' else 768, + structured_outputs=StructuredOutputsParams(json=schema, disable_additional_properties=True)) + + def generate(batch_rows): + images = [] + requests = [] + for row in batch_rows: + required = {'sample_id', 'image_path'} + if args.task == 'grefcoco': + required.add('expression') + missing = required - row.keys() + if missing: + raise ValueError(f'missing fields {sorted(missing)} for {row["sample_id"]}') + image = Image.open(row['image_path']).convert('RGB') + images.append(image) + requests.append({'prompt': make_prompt(args.root, args.model_kind, args.task, row, processor), 'multi_modal_data': {'image': image}}) + started = time.perf_counter() + generated = model.generate(requests, sampling, use_tqdm=False) + elapsed = time.perf_counter() - started + for image in images: + image.close() + if len(generated) != len(batch_rows): + raise ValueError('vLLM returned an unexpected number of results') + return elapsed, generated + + warmup_count = min(2, len(selected) - 1) + for row in selected[:warmup_count]: + generate([row]) + measured = selected[warmup_count:] + batches = [] + for start in range(0, len(measured), args.batch_size): + batch_rows = measured[start:start + args.batch_size] + batch_started = time.perf_counter() + elapsed, generated = generate(batch_rows) + wall = time.perf_counter() - batch_started + output_records = [] + for result in generated: + raw = result.outputs[0].text + error = None + try: + parsed = json.loads(raw) + jsonschema.validate(parsed, schema) + boxes = [o['bbox_2d'] for o in parsed['objects']] if args.task == 'grefcoco' else [b for e in parsed['events'].values() for b in e['boxes']] + if any(b[0] >= b[2] or b[1] >= b[3] for b in boxes): + raise ValueError('non-positive box size') + except (json.JSONDecodeError, jsonschema.ValidationError, ValueError) as exc: + parsed = None + error = str(exc) + output_records.append({'raw_response':raw,'valid_prediction':parsed,'error':error, + 'finish_reason':result.outputs[0].finish_reason,'completion_tokens':len(result.outputs[0].token_ids), + 'prompt_tokens':len(result.prompt_token_ids)}) + item = { + 'sample_ids': [row['sample_id'] for row in batch_rows], + 'batch_size': len(batch_rows), + 'generation_latency_seconds': elapsed, + 'batch_wall_latency_seconds': wall, + 'amortized_generation_ms_per_sample': elapsed * 1000 / len(batch_rows), + 'amortized_batch_wall_ms_per_sample': wall * 1000 / len(batch_rows), + 'outputs': [ + { + 'finish_reason': result.outputs[0].finish_reason, + 'stop_reason': result.outputs[0].stop_reason, + 'prompt_tokens': len(result.prompt_token_ids) if result.prompt_token_ids is not None else None, + 'completion_tokens': len(result.outputs[0].token_ids), + 'text_characters': len(result.outputs[0].text), + } + for result in generated + ], + } + batches.append(item) + item['outputs'] = output_records + args.output.parent.mkdir(parents=True, exist_ok=True) + args.output.write_text(json.dumps({ + 'status': 'running', 'model_kind': args.model_kind, 'task': args.task, 'model_path': str(model_path), + 'sample_count_requested': args.samples, 'warmup_count': warmup_count, 'batch_size': args.batch_size, + 'sample_ids': [row['sample_id'] for row in selected], 'batches': batches, + }, ensure_ascii=False, indent=2), encoding='utf-8') + print(json.dumps({'completed_samples': sum(item['batch_size'] for item in batches), 'total_measured': len(measured), 'batch_latency_seconds': elapsed}), flush=True) + all_ms = [x['amortized_generation_ms_per_sample'] for x in batches] + all_wall_ms = [x['amortized_batch_wall_ms_per_sample'] for x in batches] + output = { + 'status': 'complete', 'model_kind': args.model_kind, 'task': args.task, 'expert': expert, + 'model_path': str(model_path), 'device': 'cuda:0', 'gpu_memory_utilization': args.gpu_memory_utilization, + 'sample_count_requested': args.samples, 'warmup_count': warmup_count, 'measured_count': len(measured), + 'batch_size': args.batch_size, 'sample_ids': [row['sample_id'] for row in selected], 'batches': batches, + 'mean_generation_ms_per_sample': float(np.mean(all_ms)), 'median_generation_ms_per_sample': float(np.median(all_ms)), + 'mean_batch_wall_ms_per_sample': sum(b['batch_wall_latency_seconds'] for b in batches)*1000/len(measured), 'median_batch_wall_ms_per_sample': float(np.median(all_wall_ms)), + 'mean_prompt_tokens': float(np.mean([x['prompt_tokens'] for batch in batches for x in batch['outputs'] if x['prompt_tokens'] is not None])), + 'mean_completion_tokens': float(np.mean([x['completion_tokens'] for batch in batches for x in batch['outputs']])), + 'latency_scope': 'A800 vLLM-0.18.0 native generation latency with the frozen task prompt and structured-output schema; model loading and warmup excluded. Batch wall time includes the real request/image preparation path.', + 'flops_scope': 'This file measures latency only. Counted FLOPs are measured separately by the HF reference probe and are not claimed to be complete vLLM kernel FLOPs.', + } + args.output.write_text(json.dumps(output, ensure_ascii=False, indent=2), encoding='utf-8') + print(json.dumps({'status': 'complete', 'mean_generation_ms_per_sample': output['mean_generation_ms_per_sample'], 'mean_completion_tokens': output['mean_completion_tokens']}), flush=True) + + +if __name__ == '__main__': + main() diff --git a/paper_results/data/main_table_cost_qwen_20260912/reproducible/measure_router.py b/paper_results/data/main_table_cost_qwen_20260912/reproducible/measure_router.py new file mode 100644 index 0000000000000000000000000000000000000000..192677ef8450d4b93944d341658a5a745e82b4c8 --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/reproducible/measure_router.py @@ -0,0 +1,104 @@ +from pathlib import Path +import argparse +import json +import sys +import time +import numpy as np +import torch +from PIL import Image +from torch.utils.flop_counter import FlopCounterMode + +p=argparse.ArgumentParser() +p.add_argument('--root',type=Path,required=True) +p.add_argument('--expert',required=True) +p.add_argument('--endpoint',required=True) +p.add_argument('--output',type=Path,required=True) +a=p.parse_args() +sys.path.insert(0,str(a.root/'code')) +from routing import train_router_formal as formal +from routing.router import prepare_images,load_distilled_router +from siglip_distillation.mini_siglip import MiniSiglip +from tokenizers import Tokenizer +torch.set_num_threads(4) +torch.backends.cuda.matmul.allow_tf32=False +torch.backends.mha.set_fastpath_enabled(False) +work=a.root/'outputs/router_shared_image_hidden_20260910' +run=f'shared_12_{a.expert}_{a.endpoint}' +dest=work/'runs'/run +cfg=json.loads((dest/'config.json').read_text()) +assert cfg['seed']==42 and cfg['student']=='8M' and cfg['target']=='four_state' +records,data=formal.load_data(cfg,'test',torch.device('cuda')) +for key in ['hidden','confidence']: + norm=np.load(dest/(key+'_normalization.npz')) + data[key]=(data[key]-torch.from_numpy(norm['mean']).cuda())/torch.from_numpy(norm['std']).cuda() +model=formal.FeatureRouter(cfg,data['hidden'].shape[1],data['confidence'].shape[1],data['output'].shape[1]).cuda() +ckpt=torch.load(dest/'best.pt',map_location='cuda',weights_only=True) +model.load_state_dict(ckpt['state_dict'],strict=True) +model.eval().requires_grad_(False) +with torch.inference_mode(): + raw=torch.cat([model(data,torch.arange(i,min(i+8192,len(records)),device='cuda')).cpu() for i in range(0,len(records),8192)]).numpy() +scores=raw[:,1]-raw[:,2] +saved=np.load(work/'test_evaluation'/run/'paired_test_arrays.npz')['scores'] +reference=json.loads((work/'test_results.json').read_text()) +ref=next(g for g in reference['groups'] if g['id']==run) +point=ref['best_test_point'] +threshold=point['threshold'] +mask=scores>=threshold if point['operator']=='ge' else scores>threshold +saved_mask=saved>=threshold if point['operator']=='ge' else saved>threshold +assert np.array_equal(mask,saved_mask), 'Recomputed selected decisions differ from main-table archive' +indices=np.random.default_rng(20260912).choice(len(records),64,replace=False) +encoder_dir=a.root/'outputs/mini_siglip_strict_20260910_300ep/8M' +checkpoint=torch.load(encoder_dir/'epoch_300.pt',map_location='cpu',weights_only=True) +encoder=MiniSiglip(**checkpoint['model_config']).cuda() +encoder.load_state_dict(checkpoint['model'],strict=True) +encoder.eval().requires_grad_(False) +tokenizer=Tokenizer.from_file(str(encoder_dir/'student_tokenizer.json')) + +def prepare(i): + with Image.open(records[i]['image_path']) as im: + pixels,valid=prepare_images([[im]]) + pixels=pixels.flatten(0,1).cuda() + tokens=torch.tensor([tokenizer.encode(records[i]['expression']).ids],device='cuda') if 'text' in cfg['branches'] else None + return pixels,valid.cuda(),tokens + +def forward(i,prepared): + pixels,valid,tokens=prepared + with torch.autocast('cuda',dtype=torch.bfloat16): + image=encoder.encode_image(pixels).to(torch.float16).float().reshape(1,2,768) + text=encoder.encode_text(tokens).to(torch.float16).float() if tokens is not None else None + row={k:v[i:i+1] for k,v in data.items()} + row['image']=image;row['valid']=valid + if text is not None:row['text']=text + return model(row,torch.zeros(1,device='cuda',dtype=torch.long)) + +times=[];head_times=[];flops=[];ops=[] +with torch.inference_mode(): + for i in indices[:4]: + forward(int(i),prepare(int(i))) + torch.cuda.synchronize() + for i in indices: + i=int(i) + started=time.perf_counter() + forward(i,prepare(i)) + torch.cuda.synchronize() + times.append((time.perf_counter()-started)*1000) + start=torch.cuda.Event(enable_timing=True);end=torch.cuda.Event(enable_timing=True) + idx=torch.tensor([i],device='cuda') + start.record();model(data,idx);end.record();torch.cuda.synchronize() + head_times.append(start.elapsed_time(end)) + for i in indices[:8]: + prepared=prepare(int(i)) + with FlopCounterMode(display=False) as counter: + forward(int(i),prepared) + flops.append(counter.get_total_flops()/1e9) + ops.append({str(k):int(v) for k,v in counter.get_flop_counts().get('Global',{}).items()}) +report={'status':'complete','run':run,'config':cfg,'checkpoint':str(dest/'best.pt'), + 'score':'logit_rescue-logit_harm','test_rows_recomputed':len(records),'selected_decisions_match_main':True, + 'max_score_abs_difference':float(np.max(np.abs(saved-scores))),'actual_call_fraction':float(mask.mean()), + 'sample_count':len(indices),'sample_ids':[records[int(i)]['sample_id'] for i in indices], + 'batch_size':1,'latency_ms':times,'mean_latency_ms_per_input':float(np.mean(times)), + 'head_mean_latency_ms':float(np.mean(head_times)), 'gflops_per_input':flops,'mean_gflops_per_input':float(np.mean(flops)), + 'operators':ops,'scope':'image loading, bilinear resize, tokenize when used, two image slots (including gray placeholder), active distilled encoders, feature assembly and original routing head; cached specialist features; supported FLOPs only'} +a.output.parent.mkdir(parents=True,exist_ok=True) +a.output.write_text(json.dumps(report,indent=2),encoding='utf-8') +print(json.dumps({k:v for k,v in report.items() if k in ['run','actual_call_fraction','selected_decisions_match_main','mean_latency_ms_per_input','mean_gflops_per_input']},indent=2)) diff --git a/paper_results/data/main_table_cost_qwen_20260912/reproducible/measure_specialist.py b/paper_results/data/main_table_cost_qwen_20260912/reproducible/measure_specialist.py new file mode 100644 index 0000000000000000000000000000000000000000..fbaf817087f52f7e7dfcd6b7ac42c3d4d8360a20 --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/reproducible/measure_specialist.py @@ -0,0 +1,258 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import argparse +import importlib.util +import json +import sys +import time +from pathlib import Path + +import numpy as np +import torch +from torch.utils.flop_counter import FlopCounterMode + + +def timed_cuda(call): + start = torch.cuda.Event(enable_timing=True) + end = torch.cuda.Event(enable_timing=True) + start.record() + value = call() + end.record() + torch.cuda.synchronize() + return float(start.elapsed_time(end)), value + + +def count_flops(call): + with torch.inference_mode(), FlopCounterMode(display=False) as counter: + value = call() + torch.cuda.synchronize() + global_counts = counter.get_flop_counts().get('Global', {}) + return int(counter.get_total_flops()), value, {str(k): int(v) for k, v in global_counts.items()} + + +def instancevg_measure(root: Path, output: Path, samples: int, batch_size: int, device: str) -> None: + import mmcv + from mmcv import Config + from transformers import XLMRobertaTokenizer + from instancevg.models import build_model + from instancevg.datasets.pipelines.transforms import Normalize, Resize + + torch.set_num_threads(4) + torch.backends.cuda.matmul.allow_tf32 = False + torch.backends.cudnn.allow_tf32 = False + cfg = Config.fromfile(str(root / 'code/third_party/InstanceVG/configs/gres/InstanceVG-grefcoco.py')) + cfg.model.vis_enc.pretrain = None + cfg.model.process_visual = False + model = build_model(cfg.model) + checkpoint_path = root / 'models/grefcoco/instancevg/InstanceVG-grefcoco.pth' + checkpoint = torch.load(checkpoint_path, map_location='cpu') + state = checkpoint['state_dict'] + if all(key.startswith('module.') for key in state): + state = {key[7:]: value for key, value in state.items()} + model.load_state_dict(state, strict=True) + del state, checkpoint + model.eval().to(device).requires_grad_(False) + tokenizer_path = root / 'models/nlvr2/Raghavan--beit3_base_patch16_224_nlvr2/sentencepiece.bpe.model' + tokenizer = XLMRobertaTokenizer(str(tokenizer_path)) + resize = Resize(img_scale=(320, 320), keep_ratio=False) + normalize = Normalize(mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375]) + rows = [json.loads(line) for line in (root / 'outputs/router_full_20260910/data/instancevg/test/records.jsonl').open(encoding='utf-8')] + if samples > len(rows): + raise ValueError(f'samples {samples} exceeds {len(rows)}') + indices = np.random.default_rng(20260911).choice(len(rows), samples, replace=False) + chosen = [rows[int(index)] for index in indices] + + def prepare(batch_rows): + images, metas, tokens, masks = [], [], [], [] + for row in batch_rows: + image = mmcv.imread(row['image_path']) + if image.shape[:2] != (row['height'], row['width']): + raise ValueError(f'image shape mismatch for {row["sample_id"]}') + expression = row['expression'] + expression = __import__('re').sub(r"([.,'!?\"()*#:;])", '', expression.lower()).replace('-', ' ').replace('/', ' ') + token_ids = tokenizer.convert_tokens_to_ids(tokenizer.tokenize(expression)) + if not token_ids: + raise ValueError(f'empty token sequence for {row["sample_id"]}') + token_ids = [tokenizer.bos_token_id] + token_ids[:48] + [tokenizer.eos_token_id] + if len(token_ids) > 50: + raise ValueError(f'token sequence exceeds fixed length for {row["sample_id"]}') + masks.append([0] * len(token_ids) + [1] * (50 - len(token_ids))) + tokens.append(token_ids + [tokenizer.pad_token_id] * (50 - len(token_ids))) + data = dict(img=image, filename=row['image_path'], img_shape=image.shape, ori_shape=image.shape, + expression=expression, empty=None, with_bbox=False, with_mask=False) + data = normalize(resize(data)) + images.append(torch.from_numpy(np.ascontiguousarray(data['img'].transpose(2, 0, 1)))) + metas.append({key: data[key] for key in ['filename', 'expression', 'ori_shape', 'img_shape', 'pad_shape', 'scale_factor', 'empty']}) + image_tensor = torch.stack(images).to(device) + token_tensor = torch.tensor(tokens, device=device) + mask_tensor = torch.tensor(masks, device=device) + + def forward(): + return model(img=image_tensor, ref_expr_inds=token_tensor, text_attention_mask=mask_tensor, + img_metas=metas, return_loss=False, rescale=True, with_bbox=True, with_mask=False) + + return forward + + batches = [chosen[start:start + batch_size] for start in range(0, len(chosen), batch_size)] + prepared = [prepare(batch) for batch in batches] + for forward in prepared[:2]: + forward() + torch.cuda.synchronize() + forward_ms = [] + end_to_end_ms = [] + for batch in batches: + forward = prepare(batch) + elapsed, _ = timed_cuda(forward) + forward_ms.append(elapsed / len(batch)) + started = time.perf_counter() + forward = prepare(batch) + forward() + torch.cuda.synchronize() + end_to_end_ms.append((time.perf_counter() - started) * 1000 / len(batch)) + flop_rows = [] + for batch in batches[:min(4, len(batches))]: + forward = prepare(batch) + total, _, operators = count_flops(forward) + flop_rows.append({'batch_size': len(batch), 'flops_counted': total, 'gflops_counted_per_sample': total / len(batch) / 1e9, 'operators': operators}) + output.write_text(json.dumps({ + 'component': 'specialist', 'expert': 'instancevg', 'task': 'grefcoco', 'device': device, + 'checkpoint': str(checkpoint_path), 'sample_count': len(chosen), 'batch_size': batch_size, + 'sample_ids': [row['sample_id'] for row in chosen], + 'model_load_seconds_excluded_from_latency': True, + 'forward_latency_ms_per_sample': forward_ms, + 'end_to_end_latency_ms_per_sample': end_to_end_ms, + 'mean_forward_latency_ms_per_sample': float(np.mean(forward_ms)), + 'median_forward_latency_ms_per_sample': float(np.median(forward_ms)), + 'mean_end_to_end_latency_ms_per_sample': float(np.mean(end_to_end_ms)), + 'median_end_to_end_latency_ms_per_sample': float(np.median(end_to_end_ms)), + 'flop_profiles': flop_rows, + 'mean_counted_gflops_per_sample': float(np.mean([row['gflops_counted_per_sample'] for row in flop_rows])), + 'flops_scope': 'GPU FlopCounterMode dense products, convolutions and supported attention; multiply-add counted as 2 FLOPs. Elementwise, normalization, custom ops and postprocessing may be missing.', + 'latency_scope': 'A800 wall-clock CUDA forward latency and end-to-end latency including image read/preprocess, excluding model load; batch processing uses the recorded batch size.', + 'completed': True, + }, ensure_ascii=False, indent=2)) + + +def yolo_measure(root: Path, output: Path, samples: int, batch_size: int, device: str) -> None: + from ultralytics import YOLO + + torch.set_num_threads(4) + torch.backends.cuda.matmul.allow_tf32 = False + torch.backends.cudnn.allow_tf32 = False + checkpoint_path = root / 'models/construction/yolo26x.pt' + model = YOLO(str(checkpoint_path)) + rows = [json.loads(line) for line in (root / 'outputs/router_full_20260910/data/yolo26x/test/records.jsonl').open(encoding='utf-8')] + if samples > len(rows): + raise ValueError(f'samples {samples} exceeds {len(rows)}') + indices = np.random.default_rng(20260911).choice(len(rows), samples, replace=False) + chosen = [rows[int(index)] for index in indices] + batches = [chosen[start:start + batch_size] for start in range(0, len(chosen), batch_size)] + + def predict(batch_rows): + return model.predict(source=[row['image_path'] for row in batch_rows], imgsz=640, batch=len(batch_rows), + device=device, half=False, conf=0.001, iou=0.7, max_det=300, save=False, verbose=False) + + for batch in batches[:2]: + predict(batch) + torch.cuda.synchronize() + forward_ms, end_to_end_ms = [], [] + for batch in batches: + elapsed, _ = timed_cuda(lambda batch=batch: predict(batch)) + forward_ms.append(elapsed / len(batch)) + started = time.perf_counter() + predict(batch) + torch.cuda.synchronize() + end_to_end_ms.append((time.perf_counter() - started) * 1000 / len(batch)) + flop_rows = [] + for batch in batches[:min(4, len(batches))]: + total, _, operators = count_flops(lambda batch=batch: predict(batch)) + flop_rows.append({'batch_size': len(batch), 'flops_counted': total, 'gflops_counted_per_sample': total / len(batch) / 1e9, 'operators': operators}) + output.write_text(json.dumps({ + 'component': 'specialist', 'expert': 'yolo26x', 'task': 'construction', 'device': device, + 'checkpoint': str(checkpoint_path), 'sample_count': len(chosen), 'batch_size': batch_size, + 'sample_ids': [row['sample_id'] for row in chosen], + 'model_load_seconds_excluded_from_latency': True, + 'forward_latency_ms_per_sample': forward_ms, + 'end_to_end_latency_ms_per_sample': end_to_end_ms, + 'mean_forward_latency_ms_per_sample': float(np.mean(forward_ms)), + 'median_forward_latency_ms_per_sample': float(np.median(forward_ms)), + 'mean_end_to_end_latency_ms_per_sample': float(np.mean(end_to_end_ms)), + 'median_end_to_end_latency_ms_per_sample': float(np.median(end_to_end_ms)), + 'flop_profiles': flop_rows, + 'mean_counted_gflops_per_sample': float(np.mean([row['gflops_counted_per_sample'] for row in flop_rows])), + 'flops_scope': 'GPU FlopCounterMode around the real Ultralytics predict path; multiply-add counted as 2 FLOPs. Elementwise, postprocessing and unsupported custom ops may be missing.', + 'latency_scope': 'A800 wall-clock latency for real Ultralytics prediction including image loading/preprocess, forward and result postprocessing, excluding model load; batch processing uses the recorded batch size.', + 'completed': True, + }, ensure_ascii=False, indent=2)) + + +def router_measure(root: Path, output: Path, expert: str, endpoint: str, condition: str, seed: int, samples: int, batch_size: int, device: str) -> None: + work = root / 'outputs/router_method_improvement_20260911_r1/reproducible' + sys.path.insert(0, str(work)) + common = __import__('method_improvement_common') + spec = json.loads((root / 'outputs/router_method_improvement_20260911_r1/method_improvement_spec.json').read_text(encoding='utf-8')) + trajectory_id = f'{expert}_{endpoint}_seed{seed}' + cfg = common.make_config(spec, expert, endpoint, seed, trajectory_id) + rows, data = common.load_split(root, spec, cfg, 'test', device) + checkpoint_path = root / f'outputs/router_method_improvement_20260911_r1/conditions/{condition}_{expert}_{endpoint}_seed{seed}/best.pt' + checkpoint = torch.load(checkpoint_path, map_location=device, weights_only=True) + model = common.FeatureRouter(cfg, data['hidden'].shape[1], data['confidence'].shape[1], data['output'].shape[1]).to(device) + model.load_state_dict(checkpoint['state_dict'], strict=True) + model.eval().requires_grad_(False) + if samples > len(rows): + raise ValueError(f'samples {samples} exceeds {len(rows)}') + indices = np.random.default_rng(20260911).choice(len(rows), samples, replace=False) + chunks = [torch.as_tensor(indices[start:start + batch_size], device=device, dtype=torch.long) for start in range(0, samples, batch_size)] + for index in chunks[:2]: + model(data, index) + torch.cuda.synchronize() + latency = [] + for index in chunks: + elapsed, _ = timed_cuda(lambda index=index: model(data, index)) + latency.append(elapsed / len(index)) + index = chunks[0] + total, _, operators = count_flops(lambda: model(data, index)) + output.write_text(json.dumps({ + 'component': 'router', 'expert': expert, 'task': cfg['task'], 'endpoint': endpoint, + 'condition': condition, 'seed': seed, 'device': device, 'checkpoint': str(checkpoint_path), + 'sample_count': samples, 'batch_size': batch_size, 'row_count_loaded': len(rows), + 'mean_latency_ms_per_sample': float(np.mean(latency)), 'median_latency_ms_per_sample': float(np.median(latency)), + 'latency_ms_per_sample': latency, 'flops_counted_per_batch': total, + 'gflops_counted_per_sample': total / len(index) / 1e9, 'operators': operators, + 'flops_scope': 'GPU FlopCounterMode for the router forward; multiply-add counted as 2 FLOPs.', + 'latency_scope': 'A800 CUDA forward latency on cached router features, excluding feature loading and model load.', + 'completed': True, + }, ensure_ascii=False, indent=2)) + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument('--root', type=Path, required=True) + parser.add_argument('--output', type=Path, required=True) + parser.add_argument('--component', choices=['specialist', 'router'], required=True) + parser.add_argument('--expert', choices=['instancevg', 'yolo26x'], required=True) + parser.add_argument('--endpoint', choices=['MiniCPM', 'Qwen3.8']) + parser.add_argument('--condition', default='both') + parser.add_argument('--seed', type=int, default=42) + parser.add_argument('--samples', type=int, default=64) + parser.add_argument('--batch-size', type=int, default=8) + parser.add_argument('--device', default='cuda:0') + args = parser.parse_args() + if args.output.exists(): + raise FileExistsError(args.output) + args.output.parent.mkdir(parents=True, exist_ok=True) + if not torch.cuda.is_available(): + raise RuntimeError('CUDA is unavailable') + if args.component == 'specialist' and args.expert == 'instancevg': + instancevg_measure(args.root, args.output, args.samples, args.batch_size, args.device) + elif args.component == 'specialist': + yolo_measure(args.root, args.output, args.samples, args.batch_size, args.device) + else: + if args.endpoint is None: + raise ValueError('--endpoint is required for router') + router_measure(args.root, args.output, args.expert, args.endpoint, args.condition, args.seed, args.samples, args.batch_size, args.device) + + +if __name__ == '__main__': + main() diff --git a/paper_results/data/main_table_cost_qwen_20260912/reproducible/prepare_output.py b/paper_results/data/main_table_cost_qwen_20260912/reproducible/prepare_output.py new file mode 100644 index 0000000000000000000000000000000000000000..2f5852b9ecd89acd1afbda9caaf75a202b0fa4ed --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/reproducible/prepare_output.py @@ -0,0 +1,19 @@ +from pathlib import Path +import json +import shutil +root=Path(__file__).resolve().parents[2] +out=root/'dse_results/main_table_cost_20260912' +out.mkdir(exist_ok=True) +(out/'components').mkdir(exist_ok=True) +source=root/'paper_materials/draft_20260912_revision19/data/results.json' +data=json.loads(source.read_text(encoding='utf-8')) +groups=[g for g in data['groups'] if g['key'] in [['grefcoco','instancevg','MiniCPM'],['construction','yolo26x','Qwen3.8']]] +assert len(groups)==2 +(out/'main_table_results.json').write_text(json.dumps({'source':str(source),'protocol':data['protocol'],'groups':groups},indent=2),encoding='utf-8') +scripts=out/'reproducible' +scripts.mkdir(exist_ok=True) +for p in Path(__file__).resolve().parent.glob('*.py'): + shutil.copy2(p,scripts/p.name) +for p in Path(__file__).resolve().parent.glob('*.sh'): + shutil.copy2(p,scripts/p.name) +print(out) diff --git a/paper_results/data/main_table_cost_qwen_20260912/reproducible/read_main.py b/paper_results/data/main_table_cost_qwen_20260912/reproducible/read_main.py new file mode 100644 index 0000000000000000000000000000000000000000..d1e604ed318b6c4d5d3c4d4a9e2c1633662abcdc --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/reproducible/read_main.py @@ -0,0 +1,10 @@ +from pathlib import Path +import json +root=Path(__file__).resolve().parents[2] +data=json.loads((root/'paper_materials/draft_20260912_revision19/data/results.json').read_text(encoding='utf-8')) +print('TOP',list(data)) +for g in data['groups']: + if g['key'] in [['grefcoco','instancevg','MiniCPM'],['construction','yolo26x','Qwen3.8']]: + print(json.dumps({k:v for k,v in g.items() if k!='methods'},ensure_ascii=False)[:6000]) + for m,v in g['methods'].items(): + print(m, {k:v1 for k,v1 in v.items() if 'curve' not in k}) diff --git a/paper_results/data/main_table_cost_qwen_20260912/reproducible/setup_local.py b/paper_results/data/main_table_cost_qwen_20260912/reproducible/setup_local.py new file mode 100644 index 0000000000000000000000000000000000000000..2e533df9c9a4e92f6b0daa229e641d8f205e7919 --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/reproducible/setup_local.py @@ -0,0 +1,6 @@ +from pathlib import Path +import shutil +root=Path(__file__).resolve().parents[2] +dest=Path(__file__).resolve().parent +for source,target in [('measure_large_vllm_runtime_20260911.py','measure_large.py'),('measure_large_flops_reference_20260911.py','measure_flops.py'),('measure_expert_router_runtime_20260911.py','measure_specialist.py')]: + shutil.copy2(root/'tmp'/source,dest/target) diff --git a/paper_results/data/main_table_cost_qwen_20260912/reproducible/switch_qwen.py b/paper_results/data/main_table_cost_qwen_20260912/reproducible/switch_qwen.py new file mode 100644 index 0000000000000000000000000000000000000000..b0e16427d5d8eb9cc13b09e3ae31ad0571fec2bc --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/reproducible/switch_qwen.py @@ -0,0 +1,23 @@ +from pathlib import Path +import json +import shutil +root=Path(__file__).resolve().parents[2] +old=root/'dse_results/main_table_cost_20260912' +out=root/'dse_results/main_table_cost_qwen_20260912' +out.mkdir(exist_ok=True) +(out/'components').mkdir(exist_ok=True) +(out/'reproducible').mkdir(exist_ok=True) +(out/'logs').mkdir(exist_ok=True) +source=root/'paper_materials/draft_20260912_revision19/data/results.json' +data=json.loads(source.read_text(encoding='utf-8')) +groups=[next(g for g in data['groups'] if g['key']==key) for key in [['grefcoco','instancevg','Qwen3.8'],['construction','yolo26x','Qwen3.8']]] +(out/'main_table_results.json').write_text(json.dumps({'source':str(source),'protocol':data['protocol'],'groups':groups},indent=2),encoding='utf-8') +for name in ['instancevg_specialist.json','yolo26x_specialist.json','yolo26x_router.json','qwen_latency.json','qwen_flops.json']: + shutil.copy2(old/'components'/name,out/'components'/name) +for name in ['build_figures.py','verify_and_report.py','latex_includes.tex']: + text=(old/name).read_text(encoding='utf-8').replace("['minicpm','qwen']","['qwen_gref','qwen']").replace('gRefCOCO / M','gRefCOCO / Q').replace('InstanceVG--MiniCPM','InstanceVG--Qwen').replace('main_table_cost_20260912','main_table_cost_qwen_20260912') + (out/name).write_text(text,encoding='utf-8') +for p in (old/'reproducible').iterdir(): + if p.is_file(): shutil.copy2(p,out/'reproducible'/p.name) +for g in groups: + print(g['key'],g['run'],{m:g['methods'][m]['test_point'] for m in ['learned','confidence']}) diff --git a/paper_results/data/main_table_cost_qwen_20260912/selected_points.csv b/paper_results/data/main_table_cost_qwen_20260912/selected_points.csv new file mode 100644 index 0000000000000000000000000000000000000000..dd2dc668ab26f4f391bb1747866a2a167ad1d971 --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/selected_points.csv @@ -0,0 +1,5 @@ +pair,method,nominal_validation_budget,actual_test_call_fraction,test_metric,estimated_gflops_per_input,estimated_time_ms_per_input +grefcoco|instancevg|Qwen3.8,learned,0.56,0.5355755324277571,0.7528287440093016,17446.303931938,2243.6672827746106 +grefcoco|instancevg|Qwen3.8,confidence,0.71,0.6838612710206166,0.7243286163967898,22269.96488623913,2830.3683522642846 +construction|yolo26x|Qwen3.8,learned,0.5,0.4876830892143808,0.29875800068449215,47383.321370531434,4861.824738942901 +construction|yolo26x|Qwen3.8,confidence,0.93,0.9221038615179761,0.2597490004762105,89464.5250591521,9103.651026133362 diff --git a/paper_results/data/main_table_cost_qwen_20260912/verify_and_report.py b/paper_results/data/main_table_cost_qwen_20260912/verify_and_report.py new file mode 100644 index 0000000000000000000000000000000000000000..0b2067240b13934a01d0088d8a37d55b66773d54 --- /dev/null +++ b/paper_results/data/main_table_cost_qwen_20260912/verify_and_report.py @@ -0,0 +1,57 @@ +from pathlib import Path +import csv +import json +import fitz + +root = Path(__file__).resolve().parent +summary = json.loads((root/'cost_summary.json').read_text()) +groups = json.loads((root/'main_table_results.json').read_text())['groups'] +rows = list(csv.DictReader((root/'full_cost_performance_curves.csv').open())) +assert len(rows) == 404 +lines = ['# 主表方法:成本—性能曲线', '', '## 1 做了什么', '', + '- 使用主表原检查点、种子42及原验证集阈值;未重新训练,也未改用后续改进方法。', + '- gRefCOCO 改为 InstanceVG + Qwen,重新测量该任务上的 Qwen 成本及对应路由器;Construction 和 InstanceVG 专家复用上一轮同口径实测。', + '- 在 A800 的0、1号卡测量;每个大模型预热2张、正式测量64张,每次1张。专家和路由部分各测64张。', + '- 重算原路由头,并核对全测试集的选定阈值调用决定与主表完全一致。性能数值沿用主表固定的预测记录。', + '- 只画学习路由与置信度路由;共404个曲线点。另提供单栏四面板紧凑版。', '', + '## 2 为什么做', '', '让成本曲线与主实验比较同一个方法,避免混入后续改进结果;补计蒸馏编码器及路由头开销。', '', + '## 3 做完有什么效果', '', '| 数据集 | 方法 | 测试性能(%) | 调用率(%) | 估算GFLOPs/输入 | 估算毫秒/输入 |', + '|---|---|---:|---:|---:|---:|'] +comparisons = [] +for group in groups: + key = '|'.join(group['key']) + chosen = {p['method']:p for p in summary['selected_points'] if p['pair']==key} + for method,label in [('learned','学习路由'),('confidence','置信度路由')]: + p = chosen[method] + original = group['methods'][method]['test_point'] + assert p['test_metric'] == original['metric'] + assert p['actual_test_call_fraction'] == original['actual_fraction'] + lines.append(f"| {group['key'][0]} | {label} | {100*p['test_metric']:.2f} | {100*p['actual_test_call_fraction']:.2f} | {p['estimated_gflops_per_input']:,.1f} | {p['estimated_time_ms_per_input']:,.1f} |") + a,b = chosen['learned'],chosen['confidence'] + comparisons.append(f"{group['key'][0]}:相对置信度路由,性能提高 {100*(a['test_metric']-b['test_metric']):.2f} 个百分点,估算 FLOPs 减少 {100*(1-a['estimated_gflops_per_input']/b['estimated_gflops_per_input']):.1f}%,估算耗时减少 {100*(1-a['estimated_time_ms_per_input']/b['estimated_time_ms_per_input']):.1f}%。") +for comparison in comparisons: + lines += ['', comparison] +lines += [''] +lines += ['### 数值口径', '', + '- 每个曲线点使用验证集确定的阈值,横轴按该阈值在测试集上的实际调用率计算;圆点对应主表选定点,没有按测试集重新选点。', + '- 置信度路由成本 = 专家成本 + 实际调用率 × 大模型平均成本。学习路由再加编码器和路由头成本。', + '- 耗时为分项实测后相加的串行估算,并非各策略整套系统的直接端到端计时。加载模型和预热不计入。', + '- FLOPs 按乘加计2次统计支持的算子;大模型用8个输入估计预填充成本(含视觉编码),再加平均输出长度乘单词元解码成本。专家统计4个输入,编码器与路由头统计8个输入。', + '- 各策略共用大模型平均单次成本,尚未测量所选样本的长度差异对成本的影响;专家隐藏特征来自缓存,其额外提取开销未单独测量。置信度阈值判断开销未单独计时。', + '- 本轮单张测量与上一轮批量测量口径不同,绝对耗时不宜直接比较。', '', + '- YOLO 使用权重记录的依赖版本8.3.222,安装在任务临时目录;结束时0、1号卡显存均为0,未操作2、3号卡服务。', '', + '## 4 对应文件在哪', '', + '- `cost_compact.pdf`:推荐插入正文的单栏四面板图,宽3.44英寸、高2.05英寸。', + '- `flops_performance.pdf`、`latency_performance.pdf`:两张独立双面板图。', + '- `full_cost_performance_curves.csv`:全部404个点;`selected_points.csv`:主表4个运行点。', + '- `components/`:8份实测记录,含原始回答、错误记录及路由核验。', + '- `build_figures.py`:绘图及成本计算;`reproducible/`:远端测量脚本;`latex_includes.tex`:插入代码及图注。', + '- 远端结果:`/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/main_table_cost_qwen_20260912`。', '', + '### 检查记录', '', '已核对404个曲线点、4个主表运行点、两组全测试集调用决定及单位。PDF文字为嵌入字体,另渲染检查排版。'] +(root/'成本性能报告.md').write_text('\n'.join(lines)+'\n',encoding='utf-8') +for stem in ['cost_compact','flops_performance','latency_performance']: + doc = fitz.open(root/(stem+'.pdf')) + assert len(doc)==1 + assert all(font[1] != 'n/a' for font in doc[0].get_fonts()) + doc[0].get_pixmap(matrix=fitz.Matrix(4,4)).save(root/(stem+'_verified.png')) +print('\n'.join(lines)) diff --git "a/paper_results/data/main_table_cost_qwen_20260912/\346\210\220\346\234\254\346\200\247\350\203\275\346\212\245\345\221\212.md" "b/paper_results/data/main_table_cost_qwen_20260912/\346\210\220\346\234\254\346\200\247\350\203\275\346\212\245\345\221\212.md" new file mode 100644 index 0000000000000000000000000000000000000000..51fdd1d3a68014b53f5e26765afc5b7a13112c0d --- /dev/null +++ "b/paper_results/data/main_table_cost_qwen_20260912/\346\210\220\346\234\254\346\200\247\350\203\275\346\212\245\345\221\212.md" @@ -0,0 +1,50 @@ +# 主表方法:成本—性能曲线 + +## 1 做了什么 + +- 使用主表原检查点、种子42及原验证集阈值;未重新训练,也未改用后续改进方法。 +- gRefCOCO 改为 InstanceVG + Qwen,重新测量该任务上的 Qwen 成本及对应路由器;Construction 和 InstanceVG 专家复用上一轮同口径实测。 +- 在 A800 的0、1号卡测量;每个大模型预热2张、正式测量64张,每次1张。专家和路由部分各测64张。 +- 重算原路由头,并核对全测试集的选定阈值调用决定与主表完全一致。性能数值沿用主表固定的预测记录。 +- 只画学习路由与置信度路由;共404个曲线点。另提供单栏四面板紧凑版。 + +## 2 为什么做 + +让成本曲线与主实验比较同一个方法,避免混入后续改进结果;补计蒸馏编码器及路由头开销。 + +## 3 做完有什么效果 + +| 数据集 | 方法 | 测试性能(%) | 调用率(%) | 估算GFLOPs/输入 | 估算毫秒/输入 | +|---|---|---:|---:|---:|---:| +| grefcoco | 学习路由 | 75.28 | 53.56 | 17,446.3 | 2,243.7 | +| grefcoco | 置信度路由 | 72.43 | 68.39 | 22,270.0 | 2,830.4 | +| construction | 学习路由 | 29.88 | 48.77 | 47,383.3 | 4,861.8 | +| construction | 置信度路由 | 25.97 | 92.21 | 89,464.5 | 9,103.7 | + +grefcoco:相对置信度路由,性能提高 2.85 个百分点,估算 FLOPs 减少 21.7%,估算耗时减少 20.7%。 + +construction:相对置信度路由,性能提高 3.90 个百分点,估算 FLOPs 减少 47.0%,估算耗时减少 46.6%。 + +### 数值口径 + +- 每个曲线点使用验证集确定的阈值,横轴按该阈值在测试集上的实际调用率计算;圆点对应主表选定点,没有按测试集重新选点。 +- 置信度路由成本 = 专家成本 + 实际调用率 × 大模型平均成本。学习路由再加编码器和路由头成本。 +- 耗时为分项实测后相加的串行估算,并非各策略整套系统的直接端到端计时。加载模型和预热不计入。 +- FLOPs 按乘加计2次统计支持的算子;大模型用8个输入估计预填充成本(含视觉编码),再加平均输出长度乘单词元解码成本。专家统计4个输入,编码器与路由头统计8个输入。 +- 各策略共用大模型平均单次成本,尚未测量所选样本的长度差异对成本的影响;专家隐藏特征来自缓存,其额外提取开销未单独测量。置信度阈值判断开销未单独计时。 +- 本轮单张测量与上一轮批量测量口径不同,绝对耗时不宜直接比较。 + +- YOLO 使用权重记录的依赖版本8.3.222,安装在任务临时目录;结束时0、1号卡显存均为0,未操作2、3号卡服务。 + +## 4 对应文件在哪 + +- `cost_compact.pdf`:推荐插入正文的单栏四面板图,宽3.44英寸、高2.05英寸。 +- `flops_performance.pdf`、`latency_performance.pdf`:两张独立双面板图。 +- `full_cost_performance_curves.csv`:全部404个点;`selected_points.csv`:主表4个运行点。 +- `components/`:8份实测记录,含原始回答、错误记录及路由核验。 +- `build_figures.py`:绘图及成本计算;`reproducible/`:远端测量脚本;`latex_includes.tex`:插入代码及图注。 +- 远端结果:`/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/main_table_cost_qwen_20260912`。 + +### 检查记录 + +已核对404个曲线点、4个主表运行点、两组全测试集调用决定及单位。PDF文字为嵌入字体,另渲染检查排版。 diff --git a/paper_results/data/method_improvement_spec.json b/paper_results/data/method_improvement_spec.json new file mode 100644 index 0000000000000000000000000000000000000000..8a745d12752d9ef54290b788165fc3f86f83ccf2 --- /dev/null +++ b/paper_results/data/method_improvement_spec.json @@ -0,0 +1,114 @@ +{ + "name": "router_method_improvement_20260911", + "remote_root": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration", + "remote_work": "outputs/router_method_improvement_20260911", + "data_work": "outputs/router_full_20260910", + "student": "8M", + "student_checkpoint": "outputs/mini_siglip_strict_20260910_300ep/8M/epoch_300.pt", + "fixed": { + "fraction": 1.0, + "freeze_encoder": true, + "fusion": "concat", + "harm_cost": 1.0, + "batch_size": 256, + "weight_decay": 0.01, + "max_epochs": 50, + "patience": 8, + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90, + "cpu_threads": 1 + }, + "seeds": [42, 43, 44], + "experts": { + "cub": { + "task": "cub", + "hidden_layer": "residual_blocks__layer_15", + "hidden_depth": 16, + "branches": ["image", "hidden", "confidence", "output"], + "confidence_column": "max_probability", + "confidence_valid_column": "max_probability_valid", + "expected_rows": {"train": 4792, "val": 1200, "test": 5794} + }, + "glsim": { + "task": "cub", + "hidden_layer": "global_local_encoder__layer_11", + "hidden_depth": 12, + "branches": ["image", "hidden", "confidence", "output"], + "confidence_column": "max_probability", + "confidence_valid_column": "max_probability_valid", + "expected_rows": {"train": 4792, "val": 1200, "test": 5794} + }, + "groundingdino": { + "task": "grefcoco", + "hidden_layer": "multimodal_encoder__layer_06", + "hidden_depth": 6, + "branches": ["image", "text", "hidden", "confidence", "output"], + "confidence_column": "max_score", + "confidence_valid_column": "max_score_valid", + "expected_rows": {"train": 209344, "val": 14229, "test": 35263} + }, + "instancevg": { + "task": "grefcoco", + "hidden_layer": "joint_encoder__layer_11", + "hidden_depth": 12, + "branches": ["image", "text", "hidden", "confidence", "output"], + "confidence_column": "max_score", + "confidence_valid_column": "max_score_valid", + "expected_rows": {"train": 209344, "val": 14229, "test": 35263} + }, + "nlvr2": { + "task": "nlvr2", + "hidden_layer": "multimodal_encoder__layer_11", + "hidden_depth": 12, + "branches": ["image", "text", "hidden", "confidence", "output"], + "confidence_column": "max_probability", + "confidence_valid_column": "max_probability_valid", + "expected_rows": {"train": 85042, "val": 6982, "test": 13937} + }, + "vilt": { + "task": "nlvr2", + "hidden_layer": "ordered_image_text__layer_11", + "hidden_depth": 12, + "branches": ["image", "text", "hidden", "confidence", "output"], + "confidence_column": "max_probability", + "confidence_valid_column": "max_probability_valid", + "expected_rows": {"train": 85042, "val": 6982, "test": 13937} + }, + "yolo26x": { + "task": "construction", + "hidden_layer": "feature_blocks__layer_16", + "hidden_depth": 17, + "branches": ["image", "hidden", "confidence", "output"], + "confidence_column": "max_score", + "confidence_valid_column": "max_score_valid", + "expected_rows": {"train": 6308, "val": 701, "test": 3004} + }, + "rtdetr_x_fp32": { + "task": "construction", + "hidden_layer": "query_decoder__layer_06", + "hidden_depth": 6, + "branches": ["image", "hidden", "confidence", "output"], + "confidence_column": "max_score", + "confidence_valid_column": "max_score_valid", + "expected_rows": {"train": 6308, "val": 701, "test": 3004} + } + }, + "conditions": [ + {"name": "original", "selection_mode": "a25", "score_mode": "s_log"}, + {"name": "score_gain", "selection_mode": "a25", "score_mode": "s_gain"}, + {"name": "checkpoint_perf", "selection_mode": "max_task_performance", "score_mode": "s_log"}, + {"name": "both", "selection_mode": "max_task_performance", "score_mode": "s_gain"} + ], + "score_only_control": {"name": "score_only_gain", "source_condition": "original", "score_mode": "s_gain"}, + "budgets": { + "a25": [0.0, 0.05, 0.10, 0.15, 0.20, 0.25], + "historical_threshold": [0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.10, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.20, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.30, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.40, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.50, 0.51, 0.52, 0.53, 0.54, 0.55, 0.56, 0.57, 0.58, 0.59, 0.60, 0.61, 0.62, 0.63, 0.64, 0.65, 0.66, 0.67, 0.68, 0.69, 0.70, 0.71, 0.72, 0.73, 0.74, 0.75, 0.76, 0.77, 0.78, 0.79, 0.80, 0.81, 0.82, 0.83, 0.84, 0.85, 0.86, 0.87, 0.88, 0.89, 0.90, 0.91, 0.92, 0.93, 0.94, 0.95, 0.96, 0.97, 0.98, 0.99], + "endpoint": [0.0, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.10, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.20, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.30, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.40, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.50, 0.51, 0.52, 0.53, 0.54, 0.55, 0.56, 0.57, 0.58, 0.59, 0.60, 0.61, 0.62, 0.63, 0.64, 0.65, 0.66, 0.67, 0.68, 0.69, 0.70, 0.71, 0.72, 0.73, 0.74, 0.75, 0.76, 0.77, 0.78, 0.79, 0.80, 0.81, 0.82, 0.83, 0.84, 0.85, 0.86, 0.87, 0.88, 0.89, 0.90, 0.91, 0.92, 0.93, 0.94, 0.95, 0.96, 0.97, 0.98, 0.99, 1.0], + "batch_diagnostic": [0.0, 0.05, 0.10, 0.15, 0.20, 0.25, 0.50, 0.75, 1.0] + }, + "fusion_alphas": [0.0, 0.25, 0.5, 0.75, 1.0], + "tie_epsilon": 1e-12 +} diff --git a/paper_results/data/operation_measurements.json b/paper_results/data/operation_measurements.json new file mode 100644 index 0000000000000000000000000000000000000000..325a3b87dc347865f6656f58c6b6d6a7d8c9d925 --- /dev/null +++ b/paper_results/data/operation_measurements.json @@ -0,0 +1,129 @@ +{ + "small_models": [ + { + "model": "ResNet", + "expert": "cub", + "samples": 32, + "mean_gflops_counted": 8.175091712, + "min_gflops_counted": 8.175091712, + "max_gflops_counted": 8.175091712, + "operator_inventory_source": "cub_operator_check.json", + "operator_check_samples": 1, + "uncovered_spatial_sampling_seen": false + }, + { + "model": "YOLO26x", + "expert": "yolo26x", + "samples": 32, + "mean_gflops_counted": 143.2219032, + "min_gflops_counted": 116.2238976, + "max_gflops_counted": 193.92768, + "operator_inventory_source": "yolo26x_operator_check.json", + "operator_check_samples": 1, + "uncovered_spatial_sampling_seen": false + }, + { + "model": "RT-DETR-X", + "expert": "rtdetr_x_fp32", + "samples": 32, + "mean_gflops_counted": 224.6444544, + "min_gflops_counted": 224.6444544, + "max_gflops_counted": 224.6444544, + "operator_inventory_source": "rtdetr_x_fp32_operator_check.json", + "operator_check_samples": 1, + "uncovered_spatial_sampling_seen": true + }, + { + "model": "ViLT", + "expert": "vilt", + "samples": 32, + "mean_gflops_counted": 84.895706112, + "min_gflops_counted": 57.611655168, + "max_gflops_counted": 120.690972672, + "operator_inventory_source": "vilt_operator_check.json", + "operator_check_samples": 1, + "uncovered_spatial_sampling_seen": false + }, + { + "model": "GroundingDINO", + "expert": "groundingdino", + "samples": 32, + "mean_gflops_counted": 668.259162288, + "min_gflops_counted": 499.212779776, + "max_gflops_counted": 820.17636864, + "operator_inventory_source": "groundingdino_operator_check.json", + "operator_check_samples": 1, + "uncovered_spatial_sampling_seen": true + }, + { + "model": "BEiT3", + "expert": "nlvr2", + "samples": 32, + "mean_gflops_counted": 76.799401728, + "min_gflops_counted": 73.586743296, + "max_gflops_counted": 83.259561984, + "operator_inventory_source": "nlvr2_operator_check.json", + "operator_check_samples": 1, + "uncovered_spatial_sampling_seen": false + }, + { + "model": "GLSim", + "expert": "glsim", + "samples": 32, + "mean_gflops_counted": 70.280871936, + "min_gflops_counted": 70.280871936, + "max_gflops_counted": 70.280871936, + "operator_inventory_source": "glsim_ops.json", + "operator_check_samples": 32, + "uncovered_spatial_sampling_seen": false + } + ], + "large_input_cases": 16, + "large_input_check_complete": true, + "large_model_probes": [ + { + "model": "MiniCPM", + "sample_id": "cub-test-00517", + "task": "cub", + "prompt_tokens": 1996, + "prefill_gflops_counted": 33243.606087168, + "one_known_token_gflops_counted": 16.31253312 + }, + { + "model": "Qwen3.8", + "sample_id": "cub-test-00517", + "task": "cub", + "prompt_tokens": 4864, + "prefill_gflops_counted": 249018.23668224, + "one_known_token_gflops_counted": 53.164769472 + } + ], + "large_gpu_probe_status": { + "stage": "finished", + "occupancy_original_pid": 157057, + "stopped_process_ids": [ + 157068, + 157342, + 157343, + 157587, + 157588, + 157589, + 157590, + 157057 + ], + "started_at": 1789028871.3039298, + "measurement_exit_code": 0, + "occupancy_restored": true, + "measurement_pid": 159330, + "occupancy_new_pid": 159642, + "finished_at": 1789029043.4601417 + }, + "whole_system_flops_complete": false, + "missing_components": [ + "InstanceVG", + "full generation and all-task large-model coverage", + "uncovered operations", + "per-example routing alignment", + "95%/99% resampling intervals" + ] +} \ No newline at end of file diff --git a/paper_results/data/results.json b/paper_results/data/results.json new file mode 100644 index 0000000000000000000000000000000000000000..c3ade340d8edcff9247a2f360e5a5a5d32a5c43c --- /dev/null +++ b/paper_results/data/results.json @@ -0,0 +1,86027 @@ +{ + "protocol": "Confidence and learned routing each select a fixed threshold from the same 1..99% validation budget grid; threshold unchanged on test. No inference or training.", + "source": "C:\\Users\\csh10\\Documents\\CodexProjects\\ICASSP-COLLA\\tmp\\20260910_论文大改\\result_caches", + "groups": [ + { + "key": [ + "cub", + "cub", + "MiniCPM" + ], + "expert_display": "ResNet", + "run": "shared_12_cub_MiniCPM", + "validation_rows": 1200, + "test_rows": 5794, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/cub/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 1, + "validation_area_0_25": 0.9058333333333333, + "validation_point": { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.985, + "rescued": 4, + "harmed": 5, + "missed_rescue": 2, + "avoided_harm": 851, + "selected_both_correct": 1, + "selected_both_wrong": 2 + }, + "test_point": { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.113565757680359, + "metric": 0.799275112185019, + "rescued": 88, + "harmed": 175, + "missed_rescue": 144, + "avoided_harm": 3236, + "selected_both_correct": 56, + "selected_both_wrong": 339 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9858333333333333, + "rescued": 0, + "harmed": 0, + "missed_rescue": 6, + "avoided_harm": 856, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.985, + "rescued": 4, + "harmed": 5, + "missed_rescue": 2, + "avoided_harm": 851, + "selected_both_correct": 1, + "selected_both_wrong": 2 + }, + { + "budget": 0.02, + "threshold": -0.3627302646636963, + "operator": "ge", + "actual_fraction": 0.02, + "metric": 0.98, + "rescued": 5, + "harmed": 12, + "missed_rescue": 1, + "avoided_harm": 844, + "selected_both_correct": 2, + "selected_both_wrong": 5 + }, + { + "budget": 0.03, + "threshold": -0.4222676455974579, + "operator": "ge", + "actual_fraction": 0.03, + "metric": 0.9741666666666666, + "rescued": 5, + "harmed": 19, + "missed_rescue": 1, + "avoided_harm": 837, + "selected_both_correct": 5, + "selected_both_wrong": 7 + }, + { + "budget": 0.04, + "threshold": -0.46618229150772095, + "operator": "ge", + "actual_fraction": 0.04, + "metric": 0.9683333333333334, + "rescued": 5, + "harmed": 26, + "missed_rescue": 1, + "avoided_harm": 830, + "selected_both_correct": 8, + "selected_both_wrong": 9 + }, + { + "budget": 0.05, + "threshold": -0.5066347122192383, + "operator": "ge", + "actual_fraction": 0.05, + "metric": 0.9625, + "rescued": 5, + "harmed": 33, + "missed_rescue": 1, + "avoided_harm": 823, + "selected_both_correct": 13, + "selected_both_wrong": 9 + }, + { + "budget": 0.06, + "threshold": -0.5393391251564026, + "operator": "ge", + "actual_fraction": 0.06, + "metric": 0.9566666666666667, + "rescued": 5, + "harmed": 40, + "missed_rescue": 1, + "avoided_harm": 816, + "selected_both_correct": 16, + "selected_both_wrong": 11 + }, + { + "budget": 0.07, + "threshold": -0.5679479837417603, + "operator": "ge", + "actual_fraction": 0.07, + "metric": 0.95, + "rescued": 6, + "harmed": 49, + "missed_rescue": 0, + "avoided_harm": 807, + "selected_both_correct": 18, + "selected_both_wrong": 11 + }, + { + "budget": 0.08, + "threshold": -0.5907968282699585, + "operator": "ge", + "actual_fraction": 0.08, + "metric": 0.9416666666666667, + "rescued": 6, + "harmed": 59, + "missed_rescue": 0, + "avoided_harm": 797, + "selected_both_correct": 20, + "selected_both_wrong": 11 + }, + { + "budget": 0.09, + "threshold": -0.6168767809867859, + "operator": "ge", + "actual_fraction": 0.09, + "metric": 0.9333333333333333, + "rescued": 6, + "harmed": 69, + "missed_rescue": 0, + "avoided_harm": 787, + "selected_both_correct": 22, + "selected_both_wrong": 11 + }, + { + "budget": 0.1, + "threshold": -0.6361178159713745, + "operator": "ge", + "actual_fraction": 0.1, + "metric": 0.9258333333333333, + "rescued": 6, + "harmed": 78, + "missed_rescue": 0, + "avoided_harm": 778, + "selected_both_correct": 25, + "selected_both_wrong": 11 + }, + { + "budget": 0.11, + "threshold": -0.6547113656997681, + "operator": "ge", + "actual_fraction": 0.11, + "metric": 0.9183333333333333, + "rescued": 6, + "harmed": 87, + "missed_rescue": 0, + "avoided_harm": 769, + "selected_both_correct": 28, + "selected_both_wrong": 11 + }, + { + "budget": 0.12, + "threshold": -0.6680066585540771, + "operator": "ge", + "actual_fraction": 0.12, + "metric": 0.9091666666666667, + "rescued": 6, + "harmed": 98, + "missed_rescue": 0, + "avoided_harm": 758, + "selected_both_correct": 29, + "selected_both_wrong": 11 + }, + { + "budget": 0.13, + "threshold": -0.6809748411178589, + "operator": "ge", + "actual_fraction": 0.13, + "metric": 0.9025, + "rescued": 6, + "harmed": 106, + "missed_rescue": 0, + "avoided_harm": 750, + "selected_both_correct": 33, + "selected_both_wrong": 11 + }, + { + "budget": 0.14, + "threshold": -0.6963143348693848, + "operator": "ge", + "actual_fraction": 0.14, + "metric": 0.895, + "rescued": 6, + "harmed": 115, + "missed_rescue": 0, + "avoided_harm": 741, + "selected_both_correct": 36, + "selected_both_wrong": 11 + }, + { + "budget": 0.15, + "threshold": -0.7065573930740356, + "operator": "ge", + "actual_fraction": 0.15, + "metric": 0.8866666666666667, + "rescued": 6, + "harmed": 125, + "missed_rescue": 0, + "avoided_harm": 731, + "selected_both_correct": 38, + "selected_both_wrong": 11 + }, + { + "budget": 0.16, + "threshold": -0.7129085659980774, + "operator": "ge", + "actual_fraction": 0.16, + "metric": 0.8791666666666667, + "rescued": 6, + "harmed": 134, + "missed_rescue": 0, + "avoided_harm": 722, + "selected_both_correct": 41, + "selected_both_wrong": 11 + }, + { + "budget": 0.17, + "threshold": -0.7240402102470398, + "operator": "ge", + "actual_fraction": 0.17, + "metric": 0.8733333333333333, + "rescued": 6, + "harmed": 141, + "missed_rescue": 0, + "avoided_harm": 715, + "selected_both_correct": 46, + "selected_both_wrong": 11 + }, + { + "budget": 0.18, + "threshold": -0.7345248460769653, + "operator": "ge", + "actual_fraction": 0.18, + "metric": 0.865, + "rescued": 6, + "harmed": 151, + "missed_rescue": 0, + "avoided_harm": 705, + "selected_both_correct": 48, + "selected_both_wrong": 11 + }, + { + "budget": 0.19, + "threshold": -0.7443157434463501, + "operator": "ge", + "actual_fraction": 0.19, + "metric": 0.8591666666666666, + "rescued": 6, + "harmed": 158, + "missed_rescue": 0, + "avoided_harm": 698, + "selected_both_correct": 53, + "selected_both_wrong": 11 + }, + { + "budget": 0.2, + "threshold": -0.7533873319625854, + "operator": "ge", + "actual_fraction": 0.2, + "metric": 0.8525, + "rescued": 6, + "harmed": 166, + "missed_rescue": 0, + "avoided_harm": 690, + "selected_both_correct": 57, + "selected_both_wrong": 11 + }, + { + "budget": 0.21, + "threshold": -0.7606735825538635, + "operator": "ge", + "actual_fraction": 0.21, + "metric": 0.845, + "rescued": 6, + "harmed": 175, + "missed_rescue": 0, + "avoided_harm": 681, + "selected_both_correct": 60, + "selected_both_wrong": 11 + }, + { + "budget": 0.22, + "threshold": -0.7657199501991272, + "operator": "ge", + "actual_fraction": 0.22, + "metric": 0.8383333333333334, + "rescued": 6, + "harmed": 183, + "missed_rescue": 0, + "avoided_harm": 673, + "selected_both_correct": 64, + "selected_both_wrong": 11 + }, + { + "budget": 0.23, + "threshold": -0.7712029218673706, + "operator": "ge", + "actual_fraction": 0.23, + "metric": 0.8316666666666667, + "rescued": 6, + "harmed": 191, + "missed_rescue": 0, + "avoided_harm": 665, + "selected_both_correct": 68, + "selected_both_wrong": 11 + }, + { + "budget": 0.24, + "threshold": -0.7754930257797241, + "operator": "ge", + "actual_fraction": 0.24, + "metric": 0.825, + "rescued": 6, + "harmed": 199, + "missed_rescue": 0, + "avoided_harm": 657, + "selected_both_correct": 72, + "selected_both_wrong": 11 + }, + { + "budget": 0.25, + "threshold": -0.7811619639396667, + "operator": "ge", + "actual_fraction": 0.25, + "metric": 0.8175, + "rescued": 6, + "harmed": 208, + "missed_rescue": 0, + "avoided_harm": 648, + "selected_both_correct": 75, + "selected_both_wrong": 11 + }, + { + "budget": 0.26, + "threshold": -0.7841500043869019, + "operator": "ge", + "actual_fraction": 0.26, + "metric": 0.8108333333333333, + "rescued": 6, + "harmed": 216, + "missed_rescue": 0, + "avoided_harm": 640, + "selected_both_correct": 79, + "selected_both_wrong": 11 + }, + { + "budget": 0.27, + "threshold": -0.78773033618927, + "operator": "ge", + "actual_fraction": 0.27, + "metric": 0.8033333333333333, + "rescued": 6, + "harmed": 225, + "missed_rescue": 0, + "avoided_harm": 631, + "selected_both_correct": 82, + "selected_both_wrong": 11 + }, + { + "budget": 0.28, + "threshold": -0.7923226356506348, + "operator": "ge", + "actual_fraction": 0.28, + "metric": 0.7975, + "rescued": 6, + "harmed": 232, + "missed_rescue": 0, + "avoided_harm": 624, + "selected_both_correct": 87, + "selected_both_wrong": 11 + }, + { + "budget": 0.29, + "threshold": -0.7950519323348999, + "operator": "ge", + "actual_fraction": 0.29, + "metric": 0.79, + "rescued": 6, + "harmed": 241, + "missed_rescue": 0, + "avoided_harm": 615, + "selected_both_correct": 90, + "selected_both_wrong": 11 + }, + { + "budget": 0.3, + "threshold": -0.800890326499939, + "operator": "ge", + "actual_fraction": 0.3, + "metric": 0.7816666666666666, + "rescued": 6, + "harmed": 251, + "missed_rescue": 0, + "avoided_harm": 605, + "selected_both_correct": 92, + "selected_both_wrong": 11 + }, + { + "budget": 0.31, + "threshold": -0.8055772185325623, + "operator": "ge", + "actual_fraction": 0.31, + "metric": 0.7733333333333333, + "rescued": 6, + "harmed": 261, + "missed_rescue": 0, + "avoided_harm": 595, + "selected_both_correct": 94, + "selected_both_wrong": 11 + }, + { + "budget": 0.32, + "threshold": -0.809782087802887, + "operator": "ge", + "actual_fraction": 0.32, + "metric": 0.7675, + "rescued": 6, + "harmed": 268, + "missed_rescue": 0, + "avoided_harm": 588, + "selected_both_correct": 99, + "selected_both_wrong": 11 + }, + { + "budget": 0.33, + "threshold": -0.8134240508079529, + "operator": "ge", + "actual_fraction": 0.33, + "metric": 0.7575, + "rescued": 6, + "harmed": 280, + "missed_rescue": 0, + "avoided_harm": 576, + "selected_both_correct": 99, + "selected_both_wrong": 11 + }, + { + "budget": 0.34, + "threshold": -0.8167235255241394, + "operator": "ge", + "actual_fraction": 0.34, + "metric": 0.7516666666666667, + "rescued": 6, + "harmed": 287, + "missed_rescue": 0, + "avoided_harm": 569, + "selected_both_correct": 104, + "selected_both_wrong": 11 + }, + { + "budget": 0.35, + "threshold": -0.8208380937576294, + "operator": "ge", + "actual_fraction": 0.35, + "metric": 0.7425, + "rescued": 6, + "harmed": 298, + "missed_rescue": 0, + "avoided_harm": 558, + "selected_both_correct": 105, + "selected_both_wrong": 11 + }, + { + "budget": 0.36, + "threshold": -0.8250406384468079, + "operator": "ge", + "actual_fraction": 0.36, + "metric": 0.7375, + "rescued": 6, + "harmed": 304, + "missed_rescue": 0, + "avoided_harm": 552, + "selected_both_correct": 111, + "selected_both_wrong": 11 + }, + { + "budget": 0.37, + "threshold": -0.8295461535453796, + "operator": "ge", + "actual_fraction": 0.37, + "metric": 0.7291666666666666, + "rescued": 6, + "harmed": 314, + "missed_rescue": 0, + "avoided_harm": 542, + "selected_both_correct": 113, + "selected_both_wrong": 11 + }, + { + "budget": 0.38, + "threshold": -0.8328037261962891, + "operator": "ge", + "actual_fraction": 0.38, + "metric": 0.7191666666666666, + "rescued": 6, + "harmed": 326, + "missed_rescue": 0, + "avoided_harm": 530, + "selected_both_correct": 113, + "selected_both_wrong": 11 + }, + { + "budget": 0.39, + "threshold": -0.8359338641166687, + "operator": "ge", + "actual_fraction": 0.39, + "metric": 0.7116666666666667, + "rescued": 6, + "harmed": 335, + "missed_rescue": 0, + "avoided_harm": 521, + "selected_both_correct": 116, + "selected_both_wrong": 11 + }, + { + "budget": 0.4, + "threshold": -0.8391940593719482, + "operator": "ge", + "actual_fraction": 0.4, + "metric": 0.7033333333333334, + "rescued": 6, + "harmed": 345, + "missed_rescue": 0, + "avoided_harm": 511, + "selected_both_correct": 118, + "selected_both_wrong": 11 + }, + { + "budget": 0.41, + "threshold": -0.8424465656280518, + "operator": "ge", + "actual_fraction": 0.4091666666666667, + "metric": 0.6966666666666667, + "rescued": 6, + "harmed": 353, + "missed_rescue": 0, + "avoided_harm": 503, + "selected_both_correct": 121, + "selected_both_wrong": 11 + }, + { + "budget": 0.42, + "threshold": -0.8460794687271118, + "operator": "ge", + "actual_fraction": 0.42, + "metric": 0.6883333333333334, + "rescued": 6, + "harmed": 363, + "missed_rescue": 0, + "avoided_harm": 493, + "selected_both_correct": 124, + "selected_both_wrong": 11 + }, + { + "budget": 0.43, + "threshold": -0.8500295281410217, + "operator": "ge", + "actual_fraction": 0.43, + "metric": 0.68, + "rescued": 6, + "harmed": 373, + "missed_rescue": 0, + "avoided_harm": 483, + "selected_both_correct": 126, + "selected_both_wrong": 11 + }, + { + "budget": 0.44, + "threshold": -0.852453887462616, + "operator": "ge", + "actual_fraction": 0.44, + "metric": 0.6741666666666667, + "rescued": 6, + "harmed": 380, + "missed_rescue": 0, + "avoided_harm": 476, + "selected_both_correct": 131, + "selected_both_wrong": 11 + }, + { + "budget": 0.45, + "threshold": -0.8539360165596008, + "operator": "ge", + "actual_fraction": 0.45, + "metric": 0.6683333333333333, + "rescued": 6, + "harmed": 387, + "missed_rescue": 0, + "avoided_harm": 469, + "selected_both_correct": 136, + "selected_both_wrong": 11 + }, + { + "budget": 0.46, + "threshold": -0.8562509417533875, + "operator": "ge", + "actual_fraction": 0.46, + "metric": 0.6608333333333334, + "rescued": 6, + "harmed": 396, + "missed_rescue": 0, + "avoided_harm": 460, + "selected_both_correct": 139, + "selected_both_wrong": 11 + }, + { + "budget": 0.47, + "threshold": -0.8592684864997864, + "operator": "ge", + "actual_fraction": 0.47, + "metric": 0.6533333333333333, + "rescued": 6, + "harmed": 405, + "missed_rescue": 0, + "avoided_harm": 451, + "selected_both_correct": 142, + "selected_both_wrong": 11 + }, + { + "budget": 0.48, + "threshold": -0.8621411919593811, + "operator": "ge", + "actual_fraction": 0.48, + "metric": 0.645, + "rescued": 6, + "harmed": 415, + "missed_rescue": 0, + "avoided_harm": 441, + "selected_both_correct": 144, + "selected_both_wrong": 11 + }, + { + "budget": 0.49, + "threshold": -0.863978385925293, + "operator": "ge", + "actual_fraction": 0.49, + "metric": 0.6391666666666667, + "rescued": 6, + "harmed": 422, + "missed_rescue": 0, + "avoided_harm": 434, + "selected_both_correct": 149, + "selected_both_wrong": 11 + }, + { + "budget": 0.5, + "threshold": -0.8661247491836548, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.6341666666666667, + "rescued": 6, + "harmed": 428, + "missed_rescue": 0, + "avoided_harm": 428, + "selected_both_correct": 155, + "selected_both_wrong": 11 + }, + { + "budget": 0.51, + "threshold": -0.8688956499099731, + "operator": "ge", + "actual_fraction": 0.51, + "metric": 0.6258333333333334, + "rescued": 6, + "harmed": 438, + "missed_rescue": 0, + "avoided_harm": 418, + "selected_both_correct": 157, + "selected_both_wrong": 11 + }, + { + "budget": 0.52, + "threshold": -0.8706188201904297, + "operator": "ge", + "actual_fraction": 0.52, + "metric": 0.6191666666666666, + "rescued": 6, + "harmed": 446, + "missed_rescue": 0, + "avoided_harm": 410, + "selected_both_correct": 161, + "selected_both_wrong": 11 + }, + { + "budget": 0.53, + "threshold": -0.8727357983589172, + "operator": "ge", + "actual_fraction": 0.53, + "metric": 0.6108333333333333, + "rescued": 6, + "harmed": 456, + "missed_rescue": 0, + "avoided_harm": 400, + "selected_both_correct": 163, + "selected_both_wrong": 11 + }, + { + "budget": 0.54, + "threshold": -0.874894380569458, + "operator": "ge", + "actual_fraction": 0.54, + "metric": 0.6033333333333334, + "rescued": 6, + "harmed": 465, + "missed_rescue": 0, + "avoided_harm": 391, + "selected_both_correct": 166, + "selected_both_wrong": 11 + }, + { + "budget": 0.55, + "threshold": -0.8764559626579285, + "operator": "ge", + "actual_fraction": 0.55, + "metric": 0.5966666666666667, + "rescued": 6, + "harmed": 473, + "missed_rescue": 0, + "avoided_harm": 383, + "selected_both_correct": 170, + "selected_both_wrong": 11 + }, + { + "budget": 0.56, + "threshold": -0.879269540309906, + "operator": "ge", + "actual_fraction": 0.56, + "metric": 0.5891666666666666, + "rescued": 6, + "harmed": 482, + "missed_rescue": 0, + "avoided_harm": 374, + "selected_both_correct": 173, + "selected_both_wrong": 11 + }, + { + "budget": 0.57, + "threshold": -0.881222128868103, + "operator": "ge", + "actual_fraction": 0.5691666666666667, + "metric": 0.5816666666666667, + "rescued": 6, + "harmed": 491, + "missed_rescue": 0, + "avoided_harm": 365, + "selected_both_correct": 175, + "selected_both_wrong": 11 + }, + { + "budget": 0.58, + "threshold": -0.882451593875885, + "operator": "ge", + "actual_fraction": 0.58, + "metric": 0.575, + "rescued": 6, + "harmed": 499, + "missed_rescue": 0, + "avoided_harm": 357, + "selected_both_correct": 180, + "selected_both_wrong": 11 + }, + { + "budget": 0.59, + "threshold": -0.8843295574188232, + "operator": "ge", + "actual_fraction": 0.59, + "metric": 0.5675, + "rescued": 6, + "harmed": 508, + "missed_rescue": 0, + "avoided_harm": 348, + "selected_both_correct": 183, + "selected_both_wrong": 11 + }, + { + "budget": 0.6, + "threshold": -0.8863861560821533, + "operator": "ge", + "actual_fraction": 0.6, + "metric": 0.5608333333333333, + "rescued": 6, + "harmed": 516, + "missed_rescue": 0, + "avoided_harm": 340, + "selected_both_correct": 187, + "selected_both_wrong": 11 + }, + { + "budget": 0.61, + "threshold": -0.8876380920410156, + "operator": "ge", + "actual_fraction": 0.61, + "metric": 0.5525, + "rescued": 6, + "harmed": 526, + "missed_rescue": 0, + "avoided_harm": 330, + "selected_both_correct": 189, + "selected_both_wrong": 11 + }, + { + "budget": 0.62, + "threshold": -0.8888679146766663, + "operator": "ge", + "actual_fraction": 0.62, + "metric": 0.5466666666666666, + "rescued": 6, + "harmed": 533, + "missed_rescue": 0, + "avoided_harm": 323, + "selected_both_correct": 194, + "selected_both_wrong": 11 + }, + { + "budget": 0.63, + "threshold": -0.8905391693115234, + "operator": "ge", + "actual_fraction": 0.63, + "metric": 0.5408333333333334, + "rescued": 6, + "harmed": 540, + "missed_rescue": 0, + "avoided_harm": 316, + "selected_both_correct": 199, + "selected_both_wrong": 11 + }, + { + "budget": 0.64, + "threshold": -0.8919612169265747, + "operator": "ge", + "actual_fraction": 0.64, + "metric": 0.5325, + "rescued": 6, + "harmed": 550, + "missed_rescue": 0, + "avoided_harm": 306, + "selected_both_correct": 201, + "selected_both_wrong": 11 + }, + { + "budget": 0.65, + "threshold": -0.8940081596374512, + "operator": "ge", + "actual_fraction": 0.65, + "metric": 0.525, + "rescued": 6, + "harmed": 559, + "missed_rescue": 0, + "avoided_harm": 297, + "selected_both_correct": 204, + "selected_both_wrong": 11 + }, + { + "budget": 0.66, + "threshold": -0.895609974861145, + "operator": "ge", + "actual_fraction": 0.66, + "metric": 0.5175, + "rescued": 6, + "harmed": 568, + "missed_rescue": 0, + "avoided_harm": 288, + "selected_both_correct": 207, + "selected_both_wrong": 11 + }, + { + "budget": 0.67, + "threshold": -0.8972456455230713, + "operator": "ge", + "actual_fraction": 0.67, + "metric": 0.5091666666666667, + "rescued": 6, + "harmed": 578, + "missed_rescue": 0, + "avoided_harm": 278, + "selected_both_correct": 209, + "selected_both_wrong": 11 + }, + { + "budget": 0.68, + "threshold": -0.8986453413963318, + "operator": "ge", + "actual_fraction": 0.68, + "metric": 0.5008333333333334, + "rescued": 6, + "harmed": 588, + "missed_rescue": 0, + "avoided_harm": 268, + "selected_both_correct": 211, + "selected_both_wrong": 11 + }, + { + "budget": 0.69, + "threshold": -0.8996874094009399, + "operator": "ge", + "actual_fraction": 0.6891666666666667, + "metric": 0.49333333333333335, + "rescued": 6, + "harmed": 597, + "missed_rescue": 0, + "avoided_harm": 259, + "selected_both_correct": 213, + "selected_both_wrong": 11 + }, + { + "budget": 0.7, + "threshold": -0.9008907079696655, + "operator": "ge", + "actual_fraction": 0.7, + "metric": 0.48333333333333334, + "rescued": 6, + "harmed": 609, + "missed_rescue": 0, + "avoided_harm": 247, + "selected_both_correct": 214, + "selected_both_wrong": 11 + }, + { + "budget": 0.71, + "threshold": -0.9031620621681213, + "operator": "ge", + "actual_fraction": 0.71, + "metric": 0.47583333333333333, + "rescued": 6, + "harmed": 618, + "missed_rescue": 0, + "avoided_harm": 238, + "selected_both_correct": 217, + "selected_both_wrong": 11 + }, + { + "budget": 0.72, + "threshold": -0.9056642055511475, + "operator": "ge", + "actual_fraction": 0.72, + "metric": 0.4716666666666667, + "rescued": 6, + "harmed": 623, + "missed_rescue": 0, + "avoided_harm": 233, + "selected_both_correct": 224, + "selected_both_wrong": 11 + }, + { + "budget": 0.73, + "threshold": -0.9068957567214966, + "operator": "ge", + "actual_fraction": 0.73, + "metric": 0.465, + "rescued": 6, + "harmed": 631, + "missed_rescue": 0, + "avoided_harm": 225, + "selected_both_correct": 228, + "selected_both_wrong": 11 + }, + { + "budget": 0.74, + "threshold": -0.9093236327171326, + "operator": "ge", + "actual_fraction": 0.74, + "metric": 0.45916666666666667, + "rescued": 6, + "harmed": 638, + "missed_rescue": 0, + "avoided_harm": 218, + "selected_both_correct": 233, + "selected_both_wrong": 11 + }, + { + "budget": 0.75, + "threshold": -0.9112368822097778, + "operator": "ge", + "actual_fraction": 0.75, + "metric": 0.45166666666666666, + "rescued": 6, + "harmed": 647, + "missed_rescue": 0, + "avoided_harm": 209, + "selected_both_correct": 236, + "selected_both_wrong": 11 + }, + { + "budget": 0.76, + "threshold": -0.9129221439361572, + "operator": "ge", + "actual_fraction": 0.76, + "metric": 0.44416666666666665, + "rescued": 6, + "harmed": 656, + "missed_rescue": 0, + "avoided_harm": 200, + "selected_both_correct": 239, + "selected_both_wrong": 11 + }, + { + "budget": 0.77, + "threshold": -0.9144289493560791, + "operator": "ge", + "actual_fraction": 0.77, + "metric": 0.4375, + "rescued": 6, + "harmed": 664, + "missed_rescue": 0, + "avoided_harm": 192, + "selected_both_correct": 243, + "selected_both_wrong": 11 + }, + { + "budget": 0.78, + "threshold": -0.9158422946929932, + "operator": "ge", + "actual_fraction": 0.78, + "metric": 0.43416666666666665, + "rescued": 6, + "harmed": 668, + "missed_rescue": 0, + "avoided_harm": 188, + "selected_both_correct": 251, + "selected_both_wrong": 11 + }, + { + "budget": 0.79, + "threshold": -0.9167781472206116, + "operator": "ge", + "actual_fraction": 0.79, + "metric": 0.4266666666666667, + "rescued": 6, + "harmed": 677, + "missed_rescue": 0, + "avoided_harm": 179, + "selected_both_correct": 254, + "selected_both_wrong": 11 + }, + { + "budget": 0.8, + "threshold": -0.9187189936637878, + "operator": "ge", + "actual_fraction": 0.8, + "metric": 0.42, + "rescued": 6, + "harmed": 685, + "missed_rescue": 0, + "avoided_harm": 171, + "selected_both_correct": 258, + "selected_both_wrong": 11 + }, + { + "budget": 0.81, + "threshold": -0.9207327365875244, + "operator": "ge", + "actual_fraction": 0.81, + "metric": 0.41333333333333333, + "rescued": 6, + "harmed": 693, + "missed_rescue": 0, + "avoided_harm": 163, + "selected_both_correct": 262, + "selected_both_wrong": 11 + }, + { + "budget": 0.82, + "threshold": -0.9226345419883728, + "operator": "ge", + "actual_fraction": 0.8191666666666667, + "metric": 0.4075, + "rescued": 6, + "harmed": 700, + "missed_rescue": 0, + "avoided_harm": 156, + "selected_both_correct": 266, + "selected_both_wrong": 11 + }, + { + "budget": 0.83, + "threshold": -0.9243993759155273, + "operator": "ge", + "actual_fraction": 0.83, + "metric": 0.39916666666666667, + "rescued": 6, + "harmed": 710, + "missed_rescue": 0, + "avoided_harm": 146, + "selected_both_correct": 269, + "selected_both_wrong": 11 + }, + { + "budget": 0.84, + "threshold": -0.9264947175979614, + "operator": "ge", + "actual_fraction": 0.84, + "metric": 0.39, + "rescued": 6, + "harmed": 721, + "missed_rescue": 0, + "avoided_harm": 135, + "selected_both_correct": 270, + "selected_both_wrong": 11 + }, + { + "budget": 0.85, + "threshold": -0.9274600148200989, + "operator": "ge", + "actual_fraction": 0.85, + "metric": 0.38416666666666666, + "rescued": 6, + "harmed": 728, + "missed_rescue": 0, + "avoided_harm": 128, + "selected_both_correct": 275, + "selected_both_wrong": 11 + }, + { + "budget": 0.86, + "threshold": -0.930335283279419, + "operator": "ge", + "actual_fraction": 0.86, + "metric": 0.37833333333333335, + "rescued": 6, + "harmed": 735, + "missed_rescue": 0, + "avoided_harm": 121, + "selected_both_correct": 280, + "selected_both_wrong": 11 + }, + { + "budget": 0.87, + "threshold": -0.9319414496421814, + "operator": "ge", + "actual_fraction": 0.87, + "metric": 0.37166666666666665, + "rescued": 6, + "harmed": 743, + "missed_rescue": 0, + "avoided_harm": 113, + "selected_both_correct": 284, + "selected_both_wrong": 11 + }, + { + "budget": 0.88, + "threshold": -0.9340114593505859, + "operator": "ge", + "actual_fraction": 0.88, + "metric": 0.36583333333333334, + "rescued": 6, + "harmed": 750, + "missed_rescue": 0, + "avoided_harm": 106, + "selected_both_correct": 289, + "selected_both_wrong": 11 + }, + { + "budget": 0.89, + "threshold": -0.9363258481025696, + "operator": "ge", + "actual_fraction": 0.89, + "metric": 0.3591666666666667, + "rescued": 6, + "harmed": 758, + "missed_rescue": 0, + "avoided_harm": 98, + "selected_both_correct": 293, + "selected_both_wrong": 11 + }, + { + "budget": 0.9, + "threshold": -0.9400071501731873, + "operator": "ge", + "actual_fraction": 0.9, + "metric": 0.35, + "rescued": 6, + "harmed": 769, + "missed_rescue": 0, + "avoided_harm": 87, + "selected_both_correct": 294, + "selected_both_wrong": 11 + }, + { + "budget": 0.91, + "threshold": -0.9423198103904724, + "operator": "ge", + "actual_fraction": 0.91, + "metric": 0.3425, + "rescued": 6, + "harmed": 778, + "missed_rescue": 0, + "avoided_harm": 78, + "selected_both_correct": 297, + "selected_both_wrong": 11 + }, + { + "budget": 0.92, + "threshold": -0.9443356394767761, + "operator": "ge", + "actual_fraction": 0.92, + "metric": 0.335, + "rescued": 6, + "harmed": 787, + "missed_rescue": 0, + "avoided_harm": 69, + "selected_both_correct": 300, + "selected_both_wrong": 11 + }, + { + "budget": 0.93, + "threshold": -0.9468213319778442, + "operator": "ge", + "actual_fraction": 0.93, + "metric": 0.3283333333333333, + "rescued": 6, + "harmed": 795, + "missed_rescue": 0, + "avoided_harm": 61, + "selected_both_correct": 304, + "selected_both_wrong": 11 + }, + { + "budget": 0.94, + "threshold": -0.9496435523033142, + "operator": "ge", + "actual_fraction": 0.94, + "metric": 0.32083333333333336, + "rescued": 6, + "harmed": 804, + "missed_rescue": 0, + "avoided_harm": 52, + "selected_both_correct": 307, + "selected_both_wrong": 11 + }, + { + "budget": 0.95, + "threshold": -0.9529175758361816, + "operator": "ge", + "actual_fraction": 0.95, + "metric": 0.3125, + "rescued": 6, + "harmed": 814, + "missed_rescue": 0, + "avoided_harm": 42, + "selected_both_correct": 309, + "selected_both_wrong": 11 + }, + { + "budget": 0.96, + "threshold": -0.9560379981994629, + "operator": "ge", + "actual_fraction": 0.96, + "metric": 0.305, + "rescued": 6, + "harmed": 823, + "missed_rescue": 0, + "avoided_harm": 33, + "selected_both_correct": 312, + "selected_both_wrong": 11 + }, + { + "budget": 0.97, + "threshold": -0.9591385126113892, + "operator": "ge", + "actual_fraction": 0.97, + "metric": 0.2991666666666667, + "rescued": 6, + "harmed": 830, + "missed_rescue": 0, + "avoided_harm": 26, + "selected_both_correct": 317, + "selected_both_wrong": 11 + }, + { + "budget": 0.98, + "threshold": -0.9627243280410767, + "operator": "ge", + "actual_fraction": 0.98, + "metric": 0.2925, + "rescued": 6, + "harmed": 838, + "missed_rescue": 0, + "avoided_harm": 18, + "selected_both_correct": 321, + "selected_both_wrong": 11 + }, + { + "budget": 0.99, + "threshold": -0.9702208638191223, + "operator": "ge", + "actual_fraction": 0.99, + "metric": 0.28583333333333333, + "rescued": 6, + "harmed": 846, + "missed_rescue": 0, + "avoided_harm": 10, + "selected_both_correct": 325, + "selected_both_wrong": 11 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.2775, + "rescued": 6, + "harmed": 856, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 327, + "selected_both_wrong": 11 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.81429064549534, + "rescued": 0, + "harmed": 0, + "missed_rescue": 232, + "avoided_harm": 3411, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.113565757680359, + "metric": 0.799275112185019, + "rescued": 88, + "harmed": 175, + "missed_rescue": 144, + "avoided_harm": 3236, + "selected_both_correct": 56, + "selected_both_wrong": 339 + }, + { + "budget": 0.02, + "threshold": -0.3627302646636963, + "operator": "ge", + "actual_fraction": 0.15947531929582326, + "metric": 0.7839143942008975, + "rescued": 113, + "harmed": 289, + "missed_rescue": 119, + "avoided_harm": 3122, + "selected_both_correct": 88, + "selected_both_wrong": 434 + }, + { + "budget": 0.03, + "threshold": -0.4222676455974579, + "operator": "ge", + "actual_fraction": 0.2143596824301001, + "metric": 0.7611322057300656, + "rescued": 142, + "harmed": 450, + "missed_rescue": 90, + "avoided_harm": 2961, + "selected_both_correct": 128, + "selected_both_wrong": 522 + }, + { + "budget": 0.04, + "threshold": -0.46618229150772095, + "operator": "ge", + "actual_fraction": 0.2545736969278564, + "metric": 0.7430100103555403, + "rescued": 159, + "harmed": 572, + "missed_rescue": 73, + "avoided_harm": 2839, + "selected_both_correct": 159, + "selected_both_wrong": 585 + }, + { + "budget": 0.05, + "threshold": -0.5066347122192383, + "operator": "ge", + "actual_fraction": 0.29375215740421123, + "metric": 0.7229892992751122, + "rescued": 170, + "harmed": 699, + "missed_rescue": 62, + "avoided_harm": 2712, + "selected_both_correct": 209, + "selected_both_wrong": 624 + }, + { + "budget": 0.06, + "threshold": -0.5393391251564026, + "operator": "ge", + "actual_fraction": 0.3208491542975492, + "metric": 0.7102174663444943, + "rescued": 176, + "harmed": 779, + "missed_rescue": 56, + "avoided_harm": 2632, + "selected_both_correct": 243, + "selected_both_wrong": 661 + }, + { + "budget": 0.07, + "threshold": -0.5679479837417603, + "operator": "ge", + "actual_fraction": 0.350707628581291, + "metric": 0.6948567483603728, + "rescued": 190, + "harmed": 882, + "missed_rescue": 42, + "avoided_harm": 2529, + "selected_both_correct": 270, + "selected_both_wrong": 690 + }, + { + "budget": 0.08, + "threshold": -0.5907968282699585, + "operator": "ge", + "actual_fraction": 0.36882982395581637, + "metric": 0.684328615809458, + "rescued": 195, + "harmed": 948, + "missed_rescue": 37, + "avoided_harm": 2463, + "selected_both_correct": 289, + "selected_both_wrong": 705 + }, + { + "budget": 0.09, + "threshold": -0.6168767809867859, + "operator": "ge", + "actual_fraction": 0.391439420089748, + "metric": 0.6720745598895409, + "rescued": 201, + "harmed": 1025, + "missed_rescue": 31, + "avoided_harm": 2386, + "selected_both_correct": 319, + "selected_both_wrong": 723 + }, + { + "budget": 0.1, + "threshold": -0.6361178159713745, + "operator": "ge", + "actual_fraction": 0.4138764238867794, + "metric": 0.6601656886434243, + "rescued": 208, + "harmed": 1101, + "missed_rescue": 24, + "avoided_harm": 2310, + "selected_both_correct": 349, + "selected_both_wrong": 740 + }, + { + "budget": 0.11, + "threshold": -0.6547113656997681, + "operator": "ge", + "actual_fraction": 0.43372454263030724, + "metric": 0.6482568173973076, + "rescued": 212, + "harmed": 1174, + "missed_rescue": 20, + "avoided_harm": 2237, + "selected_both_correct": 373, + "selected_both_wrong": 754 + }, + { + "budget": 0.12, + "threshold": -0.6680066585540771, + "operator": "ge", + "actual_fraction": 0.4480497065930273, + "metric": 0.6384190541939938, + "rescued": 212, + "harmed": 1231, + "missed_rescue": 20, + "avoided_harm": 2180, + "selected_both_correct": 394, + "selected_both_wrong": 759 + }, + { + "budget": 0.13, + "threshold": -0.6809748411178589, + "operator": "ge", + "actual_fraction": 0.4630652399033483, + "metric": 0.6287538833275803, + "rescued": 214, + "harmed": 1289, + "missed_rescue": 18, + "avoided_harm": 2122, + "selected_both_correct": 412, + "selected_both_wrong": 768 + }, + { + "budget": 0.14, + "threshold": -0.6963143348693848, + "operator": "ge", + "actual_fraction": 0.479288919571971, + "metric": 0.617880566102865, + "rescued": 217, + "harmed": 1355, + "missed_rescue": 15, + "avoided_harm": 2056, + "selected_both_correct": 433, + "selected_both_wrong": 772 + }, + { + "budget": 0.15, + "threshold": -0.7065573930740356, + "operator": "ge", + "actual_fraction": 0.4932688988608906, + "metric": 0.6102865032792544, + "rescued": 220, + "harmed": 1402, + "missed_rescue": 12, + "avoided_harm": 2009, + "selected_both_correct": 457, + "selected_both_wrong": 779 + }, + { + "budget": 0.16, + "threshold": -0.7129085659980774, + "operator": "ge", + "actual_fraction": 0.5008629616845012, + "metric": 0.6054539178460476, + "rescued": 221, + "harmed": 1431, + "missed_rescue": 11, + "avoided_harm": 1980, + "selected_both_correct": 468, + "selected_both_wrong": 782 + }, + { + "budget": 0.17, + "threshold": -0.7240402102470398, + "operator": "ge", + "actual_fraction": 0.5155333103210218, + "metric": 0.5944080082844322, + "rescued": 221, + "harmed": 1495, + "missed_rescue": 11, + "avoided_harm": 1916, + "selected_both_correct": 487, + "selected_both_wrong": 784 + }, + { + "budget": 0.18, + "threshold": -0.7345248460769653, + "operator": "ge", + "actual_fraction": 0.5298584742837418, + "metric": 0.5845702450811184, + "rescued": 223, + "harmed": 1554, + "missed_rescue": 9, + "avoided_harm": 1857, + "selected_both_correct": 502, + "selected_both_wrong": 791 + }, + { + "budget": 0.19, + "threshold": -0.7443157434463501, + "operator": "ge", + "actual_fraction": 0.5455643769416638, + "metric": 0.5745598895409044, + "rescued": 224, + "harmed": 1613, + "missed_rescue": 8, + "avoided_harm": 1798, + "selected_both_correct": 529, + "selected_both_wrong": 795 + }, + { + "budget": 0.2, + "threshold": -0.7533873319625854, + "operator": "ge", + "actual_fraction": 0.5560925094925785, + "metric": 0.5674836037279944, + "rescued": 226, + "harmed": 1656, + "missed_rescue": 6, + "avoided_harm": 1755, + "selected_both_correct": 542, + "selected_both_wrong": 798 + }, + { + "budget": 0.21, + "threshold": -0.7606735825538635, + "operator": "ge", + "actual_fraction": 0.5676561960648947, + "metric": 0.5598895409043838, + "rescued": 226, + "harmed": 1700, + "missed_rescue": 6, + "avoided_harm": 1711, + "selected_both_correct": 563, + "selected_both_wrong": 800 + }, + { + "budget": 0.22, + "threshold": -0.7657199501991272, + "operator": "ge", + "actual_fraction": 0.5766309975837073, + "metric": 0.5541939937866759, + "rescued": 226, + "harmed": 1733, + "missed_rescue": 6, + "avoided_harm": 1678, + "selected_both_correct": 580, + "selected_both_wrong": 802 + }, + { + "budget": 0.23, + "threshold": -0.7712029218673706, + "operator": "ge", + "actual_fraction": 0.5861235761132205, + "metric": 0.5479806696582672, + "rescued": 226, + "harmed": 1769, + "missed_rescue": 6, + "avoided_harm": 1642, + "selected_both_correct": 597, + "selected_both_wrong": 804 + }, + { + "budget": 0.24, + "threshold": -0.7754930257797241, + "operator": "ge", + "actual_fraction": 0.5923369002416292, + "metric": 0.5434932688988608, + "rescued": 226, + "harmed": 1795, + "missed_rescue": 6, + "avoided_harm": 1616, + "selected_both_correct": 604, + "selected_both_wrong": 807 + }, + { + "budget": 0.25, + "threshold": -0.7811619639396667, + "operator": "ge", + "actual_fraction": 0.6013117017604418, + "metric": 0.5367621677597515, + "rescued": 226, + "harmed": 1834, + "missed_rescue": 6, + "avoided_harm": 1577, + "selected_both_correct": 616, + "selected_both_wrong": 808 + }, + { + "budget": 0.26, + "threshold": -0.7841500043869019, + "operator": "ge", + "actual_fraction": 0.6052813255091474, + "metric": 0.5345184673800484, + "rescued": 226, + "harmed": 1847, + "missed_rescue": 6, + "avoided_harm": 1564, + "selected_both_correct": 625, + "selected_both_wrong": 809 + }, + { + "budget": 0.27, + "threshold": -0.78773033618927, + "operator": "ge", + "actual_fraction": 0.6123576113220573, + "metric": 0.5291681049361409, + "rescued": 227, + "harmed": 1879, + "missed_rescue": 5, + "avoided_harm": 1532, + "selected_both_correct": 632, + "selected_both_wrong": 810 + }, + { + "budget": 0.28, + "threshold": -0.7923226356506348, + "operator": "ge", + "actual_fraction": 0.6206420434932689, + "metric": 0.5241629271660339, + "rescued": 227, + "harmed": 1908, + "missed_rescue": 5, + "avoided_harm": 1503, + "selected_both_correct": 649, + "selected_both_wrong": 812 + }, + { + "budget": 0.29, + "threshold": -0.7950519323348999, + "operator": "ge", + "actual_fraction": 0.6251294442526751, + "metric": 0.52140144977563, + "rescued": 227, + "harmed": 1924, + "missed_rescue": 5, + "avoided_harm": 1487, + "selected_both_correct": 658, + "selected_both_wrong": 813 + }, + { + "budget": 0.3, + "threshold": -0.800890326499939, + "operator": "ge", + "actual_fraction": 0.6373835001725924, + "metric": 0.5117362789092165, + "rescued": 227, + "harmed": 1980, + "missed_rescue": 5, + "avoided_harm": 1431, + "selected_both_correct": 672, + "selected_both_wrong": 814 + }, + { + "budget": 0.31, + "threshold": -0.8055772185325623, + "operator": "ge", + "actual_fraction": 0.6470486710390059, + "metric": 0.5055229547808078, + "rescued": 227, + "harmed": 2016, + "missed_rescue": 5, + "avoided_harm": 1395, + "selected_both_correct": 691, + "selected_both_wrong": 815 + }, + { + "budget": 0.32, + "threshold": -0.809782087802887, + "operator": "ge", + "actual_fraction": 0.6556782878840179, + "metric": 0.5005177770107008, + "rescued": 227, + "harmed": 2045, + "missed_rescue": 5, + "avoided_harm": 1366, + "selected_both_correct": 708, + "selected_both_wrong": 819 + }, + { + "budget": 0.33, + "threshold": -0.8134240508079529, + "operator": "ge", + "actual_fraction": 0.6636175353814291, + "metric": 0.4955125992405937, + "rescued": 227, + "harmed": 2074, + "missed_rescue": 5, + "avoided_harm": 1337, + "selected_both_correct": 724, + "selected_both_wrong": 820 + }, + { + "budget": 0.34, + "threshold": -0.8167235255241394, + "operator": "ge", + "actual_fraction": 0.6708664135312392, + "metric": 0.489989644459786, + "rescued": 227, + "harmed": 2106, + "missed_rescue": 5, + "avoided_harm": 1305, + "selected_both_correct": 733, + "selected_both_wrong": 821 + }, + { + "budget": 0.35, + "threshold": -0.8208380937576294, + "operator": "ge", + "actual_fraction": 0.6812219537452537, + "metric": 0.4830859509837763, + "rescued": 228, + "harmed": 2147, + "missed_rescue": 4, + "avoided_harm": 1264, + "selected_both_correct": 751, + "selected_both_wrong": 821 + }, + { + "budget": 0.36, + "threshold": -0.8250406384468079, + "operator": "ge", + "actual_fraction": 0.689333793579565, + "metric": 0.4777355885398688, + "rescued": 228, + "harmed": 2178, + "missed_rescue": 4, + "avoided_harm": 1233, + "selected_both_correct": 765, + "selected_both_wrong": 823 + }, + { + "budget": 0.37, + "threshold": -0.8295461535453796, + "operator": "ge", + "actual_fraction": 0.7017604418363824, + "metric": 0.468588194684156, + "rescued": 228, + "harmed": 2231, + "missed_rescue": 4, + "avoided_harm": 1180, + "selected_both_correct": 782, + "selected_both_wrong": 825 + }, + { + "budget": 0.38, + "threshold": -0.8328037261962891, + "operator": "ge", + "actual_fraction": 0.7088367276492924, + "metric": 0.463583016914049, + "rescued": 228, + "harmed": 2260, + "missed_rescue": 4, + "avoided_harm": 1151, + "selected_both_correct": 793, + "selected_both_wrong": 826 + }, + { + "budget": 0.39, + "threshold": -0.8359338641166687, + "operator": "ge", + "actual_fraction": 0.7166033828098033, + "metric": 0.4577148774594408, + "rescued": 228, + "harmed": 2294, + "missed_rescue": 4, + "avoided_harm": 1117, + "selected_both_correct": 804, + "selected_both_wrong": 826 + }, + { + "budget": 0.4, + "threshold": -0.8391940593719482, + "operator": "ge", + "actual_fraction": 0.7238522609596134, + "metric": 0.4527096996893338, + "rescued": 229, + "harmed": 2324, + "missed_rescue": 3, + "avoided_harm": 1087, + "selected_both_correct": 814, + "selected_both_wrong": 827 + }, + { + "budget": 0.41, + "threshold": -0.8424465656280518, + "operator": "ge", + "actual_fraction": 0.7302381774249224, + "metric": 0.44839489126682774, + "rescued": 229, + "harmed": 2349, + "missed_rescue": 3, + "avoided_harm": 1062, + "selected_both_correct": 825, + "selected_both_wrong": 828 + }, + { + "budget": 0.42, + "threshold": -0.8460794687271118, + "operator": "ge", + "actual_fraction": 0.7392129789437349, + "metric": 0.4418363824646186, + "rescued": 229, + "harmed": 2387, + "missed_rescue": 3, + "avoided_harm": 1024, + "selected_both_correct": 839, + "selected_both_wrong": 828 + }, + { + "budget": 0.43, + "threshold": -0.8500295281410217, + "operator": "ge", + "actual_fraction": 0.747152226441146, + "metric": 0.43596824301001036, + "rescued": 229, + "harmed": 2421, + "missed_rescue": 3, + "avoided_harm": 990, + "selected_both_correct": 851, + "selected_both_wrong": 828 + }, + { + "budget": 0.44, + "threshold": -0.852453887462616, + "operator": "ge", + "actual_fraction": 0.7523299965481532, + "metric": 0.4325163962720055, + "rescued": 229, + "harmed": 2441, + "missed_rescue": 3, + "avoided_harm": 970, + "selected_both_correct": 860, + "selected_both_wrong": 829 + }, + { + "budget": 0.45, + "threshold": -0.8539360165596008, + "operator": "ge", + "actual_fraction": 0.7562996202968588, + "metric": 0.42975491888160167, + "rescued": 229, + "harmed": 2457, + "missed_rescue": 3, + "avoided_harm": 954, + "selected_both_correct": 867, + "selected_both_wrong": 829 + }, + { + "budget": 0.46, + "threshold": -0.8562509417533875, + "operator": "ge", + "actual_fraction": 0.7613047980669658, + "metric": 0.42595788746979635, + "rescued": 229, + "harmed": 2479, + "missed_rescue": 3, + "avoided_harm": 932, + "selected_both_correct": 874, + "selected_both_wrong": 829 + }, + { + "budget": 0.47, + "threshold": -0.8592684864997864, + "operator": "ge", + "actual_fraction": 0.7668277528477736, + "metric": 0.4219882637210908, + "rescued": 229, + "harmed": 2502, + "missed_rescue": 3, + "avoided_harm": 909, + "selected_both_correct": 882, + "selected_both_wrong": 830 + }, + { + "budget": 0.48, + "threshold": -0.8621411919593811, + "operator": "ge", + "actual_fraction": 0.774249223334484, + "metric": 0.4161201242664826, + "rescued": 230, + "harmed": 2537, + "missed_rescue": 2, + "avoided_harm": 874, + "selected_both_correct": 888, + "selected_both_wrong": 831 + }, + { + "budget": 0.49, + "threshold": -0.863978385925293, + "operator": "ge", + "actual_fraction": 0.7790818087676907, + "metric": 0.41266827752847773, + "rescued": 230, + "harmed": 2557, + "missed_rescue": 2, + "avoided_harm": 854, + "selected_both_correct": 895, + "selected_both_wrong": 832 + }, + { + "budget": 0.5, + "threshold": -0.8661247491836548, + "operator": "ge", + "actual_fraction": 0.7849499482222989, + "metric": 0.40852606144287196, + "rescued": 230, + "harmed": 2581, + "missed_rescue": 2, + "avoided_harm": 830, + "selected_both_correct": 904, + "selected_both_wrong": 833 + }, + { + "budget": 0.51, + "threshold": -0.8688956499099731, + "operator": "ge", + "actual_fraction": 0.7928891957197101, + "metric": 0.402830514325164, + "rescued": 230, + "harmed": 2614, + "missed_rescue": 2, + "avoided_harm": 797, + "selected_both_correct": 916, + "selected_both_wrong": 834 + }, + { + "budget": 0.52, + "threshold": -0.8706188201904297, + "operator": "ge", + "actual_fraction": 0.7970314118053159, + "metric": 0.39972385226095963, + "rescued": 230, + "harmed": 2632, + "missed_rescue": 2, + "avoided_harm": 779, + "selected_both_correct": 922, + "selected_both_wrong": 834 + }, + { + "budget": 0.53, + "threshold": -0.8727357983589172, + "operator": "ge", + "actual_fraction": 0.8016914049016224, + "metric": 0.39748015188125646, + "rescued": 230, + "harmed": 2645, + "missed_rescue": 2, + "avoided_harm": 766, + "selected_both_correct": 936, + "selected_both_wrong": 834 + }, + { + "budget": 0.54, + "threshold": -0.874894380569458, + "operator": "ge", + "actual_fraction": 0.8070417673455299, + "metric": 0.39368312046945114, + "rescued": 230, + "harmed": 2667, + "missed_rescue": 2, + "avoided_harm": 744, + "selected_both_correct": 945, + "selected_both_wrong": 834 + }, + { + "budget": 0.55, + "threshold": -0.8764559626579285, + "operator": "ge", + "actual_fraction": 0.8120469451156369, + "metric": 0.39040386606834654, + "rescued": 230, + "harmed": 2686, + "missed_rescue": 2, + "avoided_harm": 725, + "selected_both_correct": 955, + "selected_both_wrong": 834 + }, + { + "budget": 0.56, + "threshold": -0.879269540309906, + "operator": "ge", + "actual_fraction": 0.8180876769071453, + "metric": 0.3860890576458405, + "rescued": 230, + "harmed": 2711, + "missed_rescue": 2, + "avoided_harm": 700, + "selected_both_correct": 964, + "selected_both_wrong": 835 + }, + { + "budget": 0.57, + "threshold": -0.881222128868103, + "operator": "ge", + "actual_fraction": 0.823610631687953, + "metric": 0.3833275802554367, + "rescued": 231, + "harmed": 2728, + "missed_rescue": 1, + "avoided_harm": 683, + "selected_both_correct": 976, + "selected_both_wrong": 837 + }, + { + "budget": 0.58, + "threshold": -0.882451593875885, + "operator": "ge", + "actual_fraction": 0.8272350707628582, + "metric": 0.38073869520193304, + "rescued": 231, + "harmed": 2743, + "missed_rescue": 1, + "avoided_harm": 668, + "selected_both_correct": 982, + "selected_both_wrong": 837 + }, + { + "budget": 0.59, + "threshold": -0.8843295574188232, + "operator": "ge", + "actual_fraction": 0.8313772868484639, + "metric": 0.37797721781152915, + "rescued": 232, + "harmed": 2760, + "missed_rescue": 0, + "avoided_harm": 651, + "selected_both_correct": 988, + "selected_both_wrong": 837 + }, + { + "budget": 0.6, + "threshold": -0.8863861560821533, + "operator": "ge", + "actual_fraction": 0.8355195029340697, + "metric": 0.3748705557473248, + "rescued": 232, + "harmed": 2778, + "missed_rescue": 0, + "avoided_harm": 633, + "selected_both_correct": 994, + "selected_both_wrong": 837 + }, + { + "budget": 0.61, + "threshold": -0.8876380920410156, + "operator": "ge", + "actual_fraction": 0.8391439420089748, + "metric": 0.3726268553676217, + "rescued": 232, + "harmed": 2791, + "missed_rescue": 0, + "avoided_harm": 620, + "selected_both_correct": 1002, + "selected_both_wrong": 837 + }, + { + "budget": 0.62, + "threshold": -0.8888679146766663, + "operator": "ge", + "actual_fraction": 0.8425957887469796, + "metric": 0.3702105626510183, + "rescued": 232, + "harmed": 2805, + "missed_rescue": 0, + "avoided_harm": 606, + "selected_both_correct": 1008, + "selected_both_wrong": 837 + }, + { + "budget": 0.63, + "threshold": -0.8905391693115234, + "operator": "ge", + "actual_fraction": 0.8469105971694857, + "metric": 0.3669313082499137, + "rescued": 232, + "harmed": 2824, + "missed_rescue": 0, + "avoided_harm": 587, + "selected_both_correct": 1014, + "selected_both_wrong": 837 + }, + { + "budget": 0.64, + "threshold": -0.8919612169265747, + "operator": "ge", + "actual_fraction": 0.8503624439074905, + "metric": 0.36468760787021054, + "rescued": 232, + "harmed": 2837, + "missed_rescue": 0, + "avoided_harm": 574, + "selected_both_correct": 1021, + "selected_both_wrong": 837 + }, + { + "budget": 0.65, + "threshold": -0.8940081596374512, + "operator": "ge", + "actual_fraction": 0.8560579910251985, + "metric": 0.360717984121505, + "rescued": 232, + "harmed": 2860, + "missed_rescue": 0, + "avoided_harm": 551, + "selected_both_correct": 1031, + "selected_both_wrong": 837 + }, + { + "budget": 0.66, + "threshold": -0.895609974861145, + "operator": "ge", + "actual_fraction": 0.8603727994477045, + "metric": 0.3574387297204004, + "rescued": 232, + "harmed": 2879, + "missed_rescue": 0, + "avoided_harm": 532, + "selected_both_correct": 1036, + "selected_both_wrong": 838 + }, + { + "budget": 0.67, + "threshold": -0.8972456455230713, + "operator": "ge", + "actual_fraction": 0.8643424231964101, + "metric": 0.3548498446668968, + "rescued": 232, + "harmed": 2894, + "missed_rescue": 0, + "avoided_harm": 517, + "selected_both_correct": 1043, + "selected_both_wrong": 839 + }, + { + "budget": 0.68, + "threshold": -0.8986453413963318, + "operator": "ge", + "actual_fraction": 0.8688298239558163, + "metric": 0.35174318260269244, + "rescued": 232, + "harmed": 2912, + "missed_rescue": 0, + "avoided_harm": 499, + "selected_both_correct": 1051, + "selected_both_wrong": 839 + }, + { + "budget": 0.69, + "threshold": -0.8996874094009399, + "operator": "ge", + "actual_fraction": 0.8712461166724197, + "metric": 0.3496720745598895, + "rescued": 232, + "harmed": 2924, + "missed_rescue": 0, + "avoided_harm": 487, + "selected_both_correct": 1053, + "selected_both_wrong": 839 + }, + { + "budget": 0.7, + "threshold": -0.9008907079696655, + "operator": "ge", + "actual_fraction": 0.8746979634104246, + "metric": 0.3470831895063859, + "rescued": 232, + "harmed": 2939, + "missed_rescue": 0, + "avoided_harm": 472, + "selected_both_correct": 1058, + "selected_both_wrong": 839 + }, + { + "budget": 0.71, + "threshold": -0.9031620621681213, + "operator": "ge", + "actual_fraction": 0.8817742492233345, + "metric": 0.34311356575768037, + "rescued": 232, + "harmed": 2962, + "missed_rescue": 0, + "avoided_harm": 449, + "selected_both_correct": 1076, + "selected_both_wrong": 839 + }, + { + "budget": 0.72, + "threshold": -0.9056642055511475, + "operator": "ge", + "actual_fraction": 0.8893683120469451, + "metric": 0.3384535726613738, + "rescued": 232, + "harmed": 2989, + "missed_rescue": 0, + "avoided_harm": 422, + "selected_both_correct": 1092, + "selected_both_wrong": 840 + }, + { + "budget": 0.73, + "threshold": -0.9068957567214966, + "operator": "ge", + "actual_fraction": 0.8933379357956507, + "metric": 0.3369002416292717, + "rescued": 232, + "harmed": 2998, + "missed_rescue": 0, + "avoided_harm": 413, + "selected_both_correct": 1106, + "selected_both_wrong": 840 + }, + { + "budget": 0.74, + "threshold": -0.9093236327171326, + "operator": "ge", + "actual_fraction": 0.8981705212288574, + "metric": 0.3336209872281671, + "rescued": 232, + "harmed": 3017, + "missed_rescue": 0, + "avoided_harm": 394, + "selected_both_correct": 1115, + "selected_both_wrong": 840 + }, + { + "budget": 0.75, + "threshold": -0.9112368822097778, + "operator": "ge", + "actual_fraction": 0.9023127373144633, + "metric": 0.3308595098377632, + "rescued": 232, + "harmed": 3033, + "missed_rescue": 0, + "avoided_harm": 378, + "selected_both_correct": 1123, + "selected_both_wrong": 840 + }, + { + "budget": 0.76, + "threshold": -0.9129221439361572, + "operator": "ge", + "actual_fraction": 0.9081808767690714, + "metric": 0.32688988608905767, + "rescued": 232, + "harmed": 3056, + "missed_rescue": 0, + "avoided_harm": 355, + "selected_both_correct": 1133, + "selected_both_wrong": 841 + }, + { + "budget": 0.77, + "threshold": -0.9144289493560791, + "operator": "ge", + "actual_fraction": 0.9123230928546773, + "metric": 0.32361063168795307, + "rescued": 232, + "harmed": 3075, + "missed_rescue": 0, + "avoided_harm": 336, + "selected_both_correct": 1138, + "selected_both_wrong": 841 + }, + { + "budget": 0.78, + "threshold": -0.9158422946929932, + "operator": "ge", + "actual_fraction": 0.9149119779081809, + "metric": 0.3217121159820504, + "rescued": 232, + "harmed": 3086, + "missed_rescue": 0, + "avoided_harm": 325, + "selected_both_correct": 1142, + "selected_both_wrong": 841 + }, + { + "budget": 0.79, + "threshold": -0.9167781472206116, + "operator": "ge", + "actual_fraction": 0.9168104936140835, + "metric": 0.32033137728684846, + "rescued": 232, + "harmed": 3094, + "missed_rescue": 0, + "avoided_harm": 317, + "selected_both_correct": 1145, + "selected_both_wrong": 841 + }, + { + "budget": 0.8, + "threshold": -0.9187189936637878, + "operator": "ge", + "actual_fraction": 0.922851225405592, + "metric": 0.3161891612012427, + "rescued": 232, + "harmed": 3118, + "missed_rescue": 0, + "avoided_harm": 293, + "selected_both_correct": 1156, + "selected_both_wrong": 841 + }, + { + "budget": 0.81, + "threshold": -0.9207327365875244, + "operator": "ge", + "actual_fraction": 0.9280289955125992, + "metric": 0.3127373144632378, + "rescued": 232, + "harmed": 3138, + "missed_rescue": 0, + "avoided_harm": 273, + "selected_both_correct": 1166, + "selected_both_wrong": 841 + }, + { + "budget": 0.82, + "threshold": -0.9226345419883728, + "operator": "ge", + "actual_fraction": 0.9342423196410079, + "metric": 0.3079047290300311, + "rescued": 232, + "harmed": 3166, + "missed_rescue": 0, + "avoided_harm": 245, + "selected_both_correct": 1174, + "selected_both_wrong": 841 + }, + { + "budget": 0.83, + "threshold": -0.9243993759155273, + "operator": "ge", + "actual_fraction": 0.9387297204004142, + "metric": 0.30479806696582673, + "rescued": 232, + "harmed": 3184, + "missed_rescue": 0, + "avoided_harm": 227, + "selected_both_correct": 1182, + "selected_both_wrong": 841 + }, + { + "budget": 0.84, + "threshold": -0.9264947175979614, + "operator": "ge", + "actual_fraction": 0.9440800828443218, + "metric": 0.3015188125647221, + "rescued": 232, + "harmed": 3203, + "missed_rescue": 0, + "avoided_harm": 208, + "selected_both_correct": 1194, + "selected_both_wrong": 841 + }, + { + "budget": 0.85, + "threshold": -0.9274600148200989, + "operator": "ge", + "actual_fraction": 0.9463237832240249, + "metric": 0.2999654815326199, + "rescued": 232, + "harmed": 3212, + "missed_rescue": 0, + "avoided_harm": 199, + "selected_both_correct": 1198, + "selected_both_wrong": 841 + }, + { + "budget": 0.86, + "threshold": -0.930335283279419, + "operator": "ge", + "actual_fraction": 0.9534000690369347, + "metric": 0.29530548843631343, + "rescued": 232, + "harmed": 3239, + "missed_rescue": 0, + "avoided_harm": 172, + "selected_both_correct": 1211, + "selected_both_wrong": 842 + }, + { + "budget": 0.87, + "threshold": -0.9319414496421814, + "operator": "ge", + "actual_fraction": 0.9558163617535381, + "metric": 0.2939247497411115, + "rescued": 232, + "harmed": 3247, + "missed_rescue": 0, + "avoided_harm": 164, + "selected_both_correct": 1217, + "selected_both_wrong": 842 + }, + { + "budget": 0.88, + "threshold": -0.9340114593505859, + "operator": "ge", + "actual_fraction": 0.958577839143942, + "metric": 0.29202623403520883, + "rescued": 232, + "harmed": 3258, + "missed_rescue": 0, + "avoided_harm": 153, + "selected_both_correct": 1222, + "selected_both_wrong": 842 + }, + { + "budget": 0.89, + "threshold": -0.9363258481025696, + "operator": "ge", + "actual_fraction": 0.9628926475664481, + "metric": 0.28874697963410423, + "rescued": 232, + "harmed": 3277, + "missed_rescue": 0, + "avoided_harm": 134, + "selected_both_correct": 1228, + "selected_both_wrong": 842 + }, + { + "budget": 0.9, + "threshold": -0.9400071501731873, + "operator": "ge", + "actual_fraction": 0.9694511563686572, + "metric": 0.2839143942008975, + "rescued": 232, + "harmed": 3305, + "missed_rescue": 0, + "avoided_harm": 106, + "selected_both_correct": 1238, + "selected_both_wrong": 842 + }, + { + "budget": 0.91, + "threshold": -0.9423198103904724, + "operator": "ge", + "actual_fraction": 0.9734207801173628, + "metric": 0.2823610631687953, + "rescued": 232, + "harmed": 3314, + "missed_rescue": 0, + "avoided_harm": 97, + "selected_both_correct": 1251, + "selected_both_wrong": 843 + }, + { + "budget": 0.92, + "threshold": -0.9443356394767761, + "operator": "ge", + "actual_fraction": 0.9761822575077667, + "metric": 0.2806351397997929, + "rescued": 232, + "harmed": 3324, + "missed_rescue": 0, + "avoided_harm": 87, + "selected_both_correct": 1256, + "selected_both_wrong": 844 + }, + { + "budget": 0.93, + "threshold": -0.9468213319778442, + "operator": "ge", + "actual_fraction": 0.9803244735933725, + "metric": 0.27839143942008976, + "rescued": 232, + "harmed": 3337, + "missed_rescue": 0, + "avoided_harm": 74, + "selected_both_correct": 1267, + "selected_both_wrong": 844 + }, + { + "budget": 0.94, + "threshold": -0.9496435523033142, + "operator": "ge", + "actual_fraction": 0.9837763203313773, + "metric": 0.27632033137728684, + "rescued": 232, + "harmed": 3349, + "missed_rescue": 0, + "avoided_harm": 62, + "selected_both_correct": 1275, + "selected_both_wrong": 844 + }, + { + "budget": 0.95, + "threshold": -0.9529175758361816, + "operator": "ge", + "actual_fraction": 0.9860200207110804, + "metric": 0.2752847773558854, + "rescued": 232, + "harmed": 3355, + "missed_rescue": 0, + "avoided_harm": 56, + "selected_both_correct": 1282, + "selected_both_wrong": 844 + }, + { + "budget": 0.96, + "threshold": -0.9560379981994629, + "operator": "ge", + "actual_fraction": 0.988608905764584, + "metric": 0.27338626164998275, + "rescued": 232, + "harmed": 3366, + "missed_rescue": 0, + "avoided_harm": 45, + "selected_both_correct": 1286, + "selected_both_wrong": 844 + }, + { + "budget": 0.97, + "threshold": -0.9591385126113892, + "operator": "ge", + "actual_fraction": 0.9920607525025888, + "metric": 0.2711425612702796, + "rescued": 232, + "harmed": 3379, + "missed_rescue": 0, + "avoided_harm": 32, + "selected_both_correct": 1293, + "selected_both_wrong": 844 + }, + { + "budget": 0.98, + "threshold": -0.9627243280410767, + "operator": "ge", + "actual_fraction": 0.9946496375560925, + "metric": 0.2697618225750777, + "rescued": 232, + "harmed": 3387, + "missed_rescue": 0, + "avoided_harm": 24, + "selected_both_correct": 1300, + "selected_both_wrong": 844 + }, + { + "budget": 0.99, + "threshold": -0.9702208638191223, + "operator": "ge", + "actual_fraction": 0.9972385226095961, + "metric": 0.2676907145322748, + "rescued": 232, + "harmed": 3399, + "missed_rescue": 0, + "avoided_harm": 12, + "selected_both_correct": 1303, + "selected_both_wrong": 844 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.26561960648947186, + "rescued": 232, + "harmed": 3411, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 1307, + "selected_both_wrong": 844 + } + ] + }, + "learned": { + "selected_budget_index": 1, + "validation_area_0_25": 0.9621666666666666, + "validation_point": { + "budget": 0.01, + "threshold": -0.590226411819458, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9858333333333333, + "rescued": 1, + "harmed": 1, + "missed_rescue": 5, + "avoided_harm": 855, + "selected_both_correct": 8, + "selected_both_wrong": 2 + }, + "test_point": { + "budget": 0.01, + "threshold": -0.590226411819458, + "operator": "ge", + "actual_fraction": 0.028822920262340353, + "metric": 0.8137728684846393, + "rescued": 11, + "harmed": 14, + "missed_rescue": 221, + "avoided_harm": 3397, + "selected_both_correct": 80, + "selected_both_wrong": 62 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9858333333333333, + "rescued": 0, + "harmed": 0, + "missed_rescue": 6, + "avoided_harm": 856, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.590226411819458, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9858333333333333, + "rescued": 1, + "harmed": 1, + "missed_rescue": 5, + "avoided_harm": 855, + "selected_both_correct": 8, + "selected_both_wrong": 2 + }, + { + "budget": 0.02, + "threshold": -1.8696393966674805, + "operator": "ge", + "actual_fraction": 0.02, + "metric": 0.9841666666666666, + "rescued": 1, + "harmed": 3, + "missed_rescue": 5, + "avoided_harm": 853, + "selected_both_correct": 18, + "selected_both_wrong": 2 + }, + { + "budget": 0.03, + "threshold": -2.8970229625701904, + "operator": "ge", + "actual_fraction": 0.03, + "metric": 0.9825, + "rescued": 1, + "harmed": 5, + "missed_rescue": 5, + "avoided_harm": 851, + "selected_both_correct": 28, + "selected_both_wrong": 2 + }, + { + "budget": 0.04, + "threshold": -3.5405211448669434, + "operator": "ge", + "actual_fraction": 0.04, + "metric": 0.9808333333333333, + "rescued": 1, + "harmed": 7, + "missed_rescue": 5, + "avoided_harm": 849, + "selected_both_correct": 38, + "selected_both_wrong": 2 + }, + { + "budget": 0.05, + "threshold": -4.489654064178467, + "operator": "ge", + "actual_fraction": 0.05, + "metric": 0.9791666666666666, + "rescued": 1, + "harmed": 9, + "missed_rescue": 5, + "avoided_harm": 847, + "selected_both_correct": 48, + "selected_both_wrong": 2 + }, + { + "budget": 0.06, + "threshold": -4.871606826782227, + "operator": "ge", + "actual_fraction": 0.06, + "metric": 0.98, + "rescued": 2, + "harmed": 9, + "missed_rescue": 4, + "avoided_harm": 847, + "selected_both_correct": 59, + "selected_both_wrong": 2 + }, + { + "budget": 0.07, + "threshold": -5.445620059967041, + "operator": "ge", + "actual_fraction": 0.07, + "metric": 0.9783333333333334, + "rescued": 2, + "harmed": 11, + "missed_rescue": 4, + "avoided_harm": 845, + "selected_both_correct": 69, + "selected_both_wrong": 2 + }, + { + "budget": 0.08, + "threshold": -5.9812140464782715, + "operator": "ge", + "actual_fraction": 0.08, + "metric": 0.9766666666666667, + "rescued": 2, + "harmed": 13, + "missed_rescue": 4, + "avoided_harm": 843, + "selected_both_correct": 79, + "selected_both_wrong": 2 + }, + { + "budget": 0.09, + "threshold": -6.437536239624023, + "operator": "ge", + "actual_fraction": 0.09, + "metric": 0.9766666666666667, + "rescued": 2, + "harmed": 13, + "missed_rescue": 4, + "avoided_harm": 843, + "selected_both_correct": 90, + "selected_both_wrong": 3 + }, + { + "budget": 0.1, + "threshold": -6.952746868133545, + "operator": "ge", + "actual_fraction": 0.1, + "metric": 0.9741666666666666, + "rescued": 2, + "harmed": 16, + "missed_rescue": 4, + "avoided_harm": 840, + "selected_both_correct": 98, + "selected_both_wrong": 4 + }, + { + "budget": 0.11, + "threshold": -7.571210861206055, + "operator": "ge", + "actual_fraction": 0.11, + "metric": 0.97, + "rescued": 2, + "harmed": 21, + "missed_rescue": 4, + "avoided_harm": 835, + "selected_both_correct": 105, + "selected_both_wrong": 4 + }, + { + "budget": 0.12, + "threshold": -7.767908573150635, + "operator": "ge", + "actual_fraction": 0.12, + "metric": 0.9658333333333333, + "rescued": 2, + "harmed": 26, + "missed_rescue": 4, + "avoided_harm": 830, + "selected_both_correct": 111, + "selected_both_wrong": 5 + }, + { + "budget": 0.13, + "threshold": -8.024438858032227, + "operator": "ge", + "actual_fraction": 0.13, + "metric": 0.9625, + "rescued": 2, + "harmed": 30, + "missed_rescue": 4, + "avoided_harm": 826, + "selected_both_correct": 119, + "selected_both_wrong": 5 + }, + { + "budget": 0.14, + "threshold": -8.368799209594727, + "operator": "ge", + "actual_fraction": 0.14, + "metric": 0.9616666666666667, + "rescued": 2, + "harmed": 31, + "missed_rescue": 4, + "avoided_harm": 825, + "selected_both_correct": 130, + "selected_both_wrong": 5 + }, + { + "budget": 0.15, + "threshold": -8.57835578918457, + "operator": "ge", + "actual_fraction": 0.15, + "metric": 0.9591666666666666, + "rescued": 2, + "harmed": 34, + "missed_rescue": 4, + "avoided_harm": 822, + "selected_both_correct": 139, + "selected_both_wrong": 5 + }, + { + "budget": 0.16, + "threshold": -8.983278274536133, + "operator": "ge", + "actual_fraction": 0.16, + "metric": 0.955, + "rescued": 2, + "harmed": 39, + "missed_rescue": 4, + "avoided_harm": 817, + "selected_both_correct": 146, + "selected_both_wrong": 5 + }, + { + "budget": 0.17, + "threshold": -9.323188781738281, + "operator": "ge", + "actual_fraction": 0.17, + "metric": 0.9516666666666667, + "rescued": 2, + "harmed": 43, + "missed_rescue": 4, + "avoided_harm": 813, + "selected_both_correct": 154, + "selected_both_wrong": 5 + }, + { + "budget": 0.18, + "threshold": -9.623194694519043, + "operator": "ge", + "actual_fraction": 0.18, + "metric": 0.95, + "rescued": 2, + "harmed": 45, + "missed_rescue": 4, + "avoided_harm": 811, + "selected_both_correct": 164, + "selected_both_wrong": 5 + }, + { + "budget": 0.19, + "threshold": -9.886858940124512, + "operator": "ge", + "actual_fraction": 0.19, + "metric": 0.9475, + "rescued": 3, + "harmed": 49, + "missed_rescue": 3, + "avoided_harm": 807, + "selected_both_correct": 171, + "selected_both_wrong": 5 + }, + { + "budget": 0.2, + "threshold": -10.168641090393066, + "operator": "ge", + "actual_fraction": 0.2, + "metric": 0.945, + "rescued": 3, + "harmed": 52, + "missed_rescue": 3, + "avoided_harm": 804, + "selected_both_correct": 179, + "selected_both_wrong": 6 + }, + { + "budget": 0.21, + "threshold": -10.403701782226562, + "operator": "ge", + "actual_fraction": 0.21, + "metric": 0.9391666666666667, + "rescued": 3, + "harmed": 59, + "missed_rescue": 3, + "avoided_harm": 797, + "selected_both_correct": 184, + "selected_both_wrong": 6 + }, + { + "budget": 0.22, + "threshold": -10.783614158630371, + "operator": "ge", + "actual_fraction": 0.22, + "metric": 0.935, + "rescued": 3, + "harmed": 64, + "missed_rescue": 3, + "avoided_harm": 792, + "selected_both_correct": 191, + "selected_both_wrong": 6 + }, + { + "budget": 0.23, + "threshold": -11.247787475585938, + "operator": "ge", + "actual_fraction": 0.23, + "metric": 0.9308333333333333, + "rescued": 3, + "harmed": 69, + "missed_rescue": 3, + "avoided_harm": 787, + "selected_both_correct": 197, + "selected_both_wrong": 7 + }, + { + "budget": 0.24, + "threshold": -11.575923919677734, + "operator": "ge", + "actual_fraction": 0.24, + "metric": 0.9266666666666666, + "rescued": 3, + "harmed": 74, + "missed_rescue": 3, + "avoided_harm": 782, + "selected_both_correct": 204, + "selected_both_wrong": 7 + }, + { + "budget": 0.25, + "threshold": -12.19352912902832, + "operator": "ge", + "actual_fraction": 0.25, + "metric": 0.9208333333333333, + "rescued": 3, + "harmed": 81, + "missed_rescue": 3, + "avoided_harm": 775, + "selected_both_correct": 209, + "selected_both_wrong": 7 + }, + { + "budget": 0.26, + "threshold": -12.560779571533203, + "operator": "ge", + "actual_fraction": 0.26, + "metric": 0.9166666666666666, + "rescued": 4, + "harmed": 87, + "missed_rescue": 2, + "avoided_harm": 769, + "selected_both_correct": 214, + "selected_both_wrong": 7 + }, + { + "budget": 0.27, + "threshold": -12.845895767211914, + "operator": "ge", + "actual_fraction": 0.27, + "metric": 0.9125, + "rescued": 5, + "harmed": 93, + "missed_rescue": 1, + "avoided_harm": 763, + "selected_both_correct": 219, + "selected_both_wrong": 7 + }, + { + "budget": 0.28, + "threshold": -13.137770652770996, + "operator": "ge", + "actual_fraction": 0.28, + "metric": 0.9066666666666666, + "rescued": 5, + "harmed": 100, + "missed_rescue": 1, + "avoided_harm": 756, + "selected_both_correct": 223, + "selected_both_wrong": 8 + }, + { + "budget": 0.29, + "threshold": -13.375581741333008, + "operator": "ge", + "actual_fraction": 0.29, + "metric": 0.9016666666666666, + "rescued": 6, + "harmed": 107, + "missed_rescue": 0, + "avoided_harm": 749, + "selected_both_correct": 227, + "selected_both_wrong": 8 + }, + { + "budget": 0.3, + "threshold": -13.533597946166992, + "operator": "ge", + "actual_fraction": 0.3, + "metric": 0.8958333333333334, + "rescued": 6, + "harmed": 114, + "missed_rescue": 0, + "avoided_harm": 742, + "selected_both_correct": 231, + "selected_both_wrong": 9 + }, + { + "budget": 0.31, + "threshold": -13.760375022888184, + "operator": "ge", + "actual_fraction": 0.31, + "metric": 0.8866666666666667, + "rescued": 6, + "harmed": 125, + "missed_rescue": 0, + "avoided_harm": 731, + "selected_both_correct": 232, + "selected_both_wrong": 9 + }, + { + "budget": 0.32, + "threshold": -13.987640380859375, + "operator": "ge", + "actual_fraction": 0.32, + "metric": 0.8791666666666667, + "rescued": 6, + "harmed": 134, + "missed_rescue": 0, + "avoided_harm": 722, + "selected_both_correct": 234, + "selected_both_wrong": 10 + }, + { + "budget": 0.33, + "threshold": -14.21721076965332, + "operator": "ge", + "actual_fraction": 0.33, + "metric": 0.8741666666666666, + "rescued": 6, + "harmed": 140, + "missed_rescue": 0, + "avoided_harm": 716, + "selected_both_correct": 240, + "selected_both_wrong": 10 + }, + { + "budget": 0.34, + "threshold": -14.470904350280762, + "operator": "ge", + "actual_fraction": 0.34, + "metric": 0.8683333333333333, + "rescued": 6, + "harmed": 147, + "missed_rescue": 0, + "avoided_harm": 709, + "selected_both_correct": 245, + "selected_both_wrong": 10 + }, + { + "budget": 0.35, + "threshold": -14.772564888000488, + "operator": "ge", + "actual_fraction": 0.35, + "metric": 0.8591666666666666, + "rescued": 6, + "harmed": 158, + "missed_rescue": 0, + "avoided_harm": 698, + "selected_both_correct": 246, + "selected_both_wrong": 10 + }, + { + "budget": 0.36, + "threshold": -15.04111099243164, + "operator": "ge", + "actual_fraction": 0.36, + "metric": 0.8508333333333333, + "rescued": 6, + "harmed": 168, + "missed_rescue": 0, + "avoided_harm": 688, + "selected_both_correct": 248, + "selected_both_wrong": 10 + }, + { + "budget": 0.37, + "threshold": -15.203546524047852, + "operator": "ge", + "actual_fraction": 0.37, + "metric": 0.8441666666666666, + "rescued": 6, + "harmed": 176, + "missed_rescue": 0, + "avoided_harm": 680, + "selected_both_correct": 252, + "selected_both_wrong": 10 + }, + { + "budget": 0.38, + "threshold": -15.385844230651855, + "operator": "ge", + "actual_fraction": 0.38, + "metric": 0.8383333333333334, + "rescued": 6, + "harmed": 183, + "missed_rescue": 0, + "avoided_harm": 673, + "selected_both_correct": 257, + "selected_both_wrong": 10 + }, + { + "budget": 0.39, + "threshold": -15.582735061645508, + "operator": "ge", + "actual_fraction": 0.39, + "metric": 0.8333333333333334, + "rescued": 6, + "harmed": 189, + "missed_rescue": 0, + "avoided_harm": 667, + "selected_both_correct": 263, + "selected_both_wrong": 10 + }, + { + "budget": 0.4, + "threshold": -15.746196746826172, + "operator": "ge", + "actual_fraction": 0.4, + "metric": 0.825, + "rescued": 6, + "harmed": 199, + "missed_rescue": 0, + "avoided_harm": 657, + "selected_both_correct": 265, + "selected_both_wrong": 10 + }, + { + "budget": 0.41, + "threshold": -15.894475936889648, + "operator": "ge", + "actual_fraction": 0.4091666666666667, + "metric": 0.8158333333333333, + "rescued": 6, + "harmed": 210, + "missed_rescue": 0, + "avoided_harm": 646, + "selected_both_correct": 265, + "selected_both_wrong": 10 + }, + { + "budget": 0.42, + "threshold": -16.114011764526367, + "operator": "ge", + "actual_fraction": 0.42, + "metric": 0.805, + "rescued": 6, + "harmed": 223, + "missed_rescue": 0, + "avoided_harm": 633, + "selected_both_correct": 265, + "selected_both_wrong": 10 + }, + { + "budget": 0.43, + "threshold": -16.257381439208984, + "operator": "ge", + "actual_fraction": 0.43, + "metric": 0.7958333333333333, + "rescued": 6, + "harmed": 234, + "missed_rescue": 0, + "avoided_harm": 622, + "selected_both_correct": 266, + "selected_both_wrong": 10 + }, + { + "budget": 0.44, + "threshold": -16.480209350585938, + "operator": "ge", + "actual_fraction": 0.44, + "metric": 0.7891666666666667, + "rescued": 6, + "harmed": 242, + "missed_rescue": 0, + "avoided_harm": 614, + "selected_both_correct": 270, + "selected_both_wrong": 10 + }, + { + "budget": 0.45, + "threshold": -16.728397369384766, + "operator": "ge", + "actual_fraction": 0.45, + "metric": 0.7816666666666666, + "rescued": 6, + "harmed": 251, + "missed_rescue": 0, + "avoided_harm": 605, + "selected_both_correct": 273, + "selected_both_wrong": 10 + }, + { + "budget": 0.46, + "threshold": -16.844228744506836, + "operator": "ge", + "actual_fraction": 0.46, + "metric": 0.775, + "rescued": 6, + "harmed": 259, + "missed_rescue": 0, + "avoided_harm": 597, + "selected_both_correct": 277, + "selected_both_wrong": 10 + }, + { + "budget": 0.47, + "threshold": -17.066635131835938, + "operator": "ge", + "actual_fraction": 0.47, + "metric": 0.765, + "rescued": 6, + "harmed": 271, + "missed_rescue": 0, + "avoided_harm": 585, + "selected_both_correct": 277, + "selected_both_wrong": 10 + }, + { + "budget": 0.48, + "threshold": -17.25113296508789, + "operator": "ge", + "actual_fraction": 0.48, + "metric": 0.755, + "rescued": 6, + "harmed": 283, + "missed_rescue": 0, + "avoided_harm": 573, + "selected_both_correct": 277, + "selected_both_wrong": 10 + }, + { + "budget": 0.49, + "threshold": -17.416290283203125, + "operator": "ge", + "actual_fraction": 0.49, + "metric": 0.7483333333333333, + "rescued": 6, + "harmed": 291, + "missed_rescue": 0, + "avoided_harm": 565, + "selected_both_correct": 280, + "selected_both_wrong": 11 + }, + { + "budget": 0.5, + "threshold": -17.648540496826172, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.7391666666666666, + "rescued": 6, + "harmed": 302, + "missed_rescue": 0, + "avoided_harm": 554, + "selected_both_correct": 281, + "selected_both_wrong": 11 + }, + { + "budget": 0.51, + "threshold": -17.833698272705078, + "operator": "ge", + "actual_fraction": 0.51, + "metric": 0.73, + "rescued": 6, + "harmed": 313, + "missed_rescue": 0, + "avoided_harm": 543, + "selected_both_correct": 282, + "selected_both_wrong": 11 + }, + { + "budget": 0.52, + "threshold": -17.978940963745117, + "operator": "ge", + "actual_fraction": 0.52, + "metric": 0.7208333333333333, + "rescued": 6, + "harmed": 324, + "missed_rescue": 0, + "avoided_harm": 532, + "selected_both_correct": 283, + "selected_both_wrong": 11 + }, + { + "budget": 0.53, + "threshold": -18.172624588012695, + "operator": "ge", + "actual_fraction": 0.53, + "metric": 0.7141666666666666, + "rescued": 6, + "harmed": 332, + "missed_rescue": 0, + "avoided_harm": 524, + "selected_both_correct": 287, + "selected_both_wrong": 11 + }, + { + "budget": 0.54, + "threshold": -18.375865936279297, + "operator": "ge", + "actual_fraction": 0.54, + "metric": 0.7066666666666667, + "rescued": 6, + "harmed": 341, + "missed_rescue": 0, + "avoided_harm": 515, + "selected_both_correct": 290, + "selected_both_wrong": 11 + }, + { + "budget": 0.55, + "threshold": -18.543420791625977, + "operator": "ge", + "actual_fraction": 0.55, + "metric": 0.6966666666666667, + "rescued": 6, + "harmed": 353, + "missed_rescue": 0, + "avoided_harm": 503, + "selected_both_correct": 290, + "selected_both_wrong": 11 + }, + { + "budget": 0.56, + "threshold": -18.668851852416992, + "operator": "ge", + "actual_fraction": 0.56, + "metric": 0.6875, + "rescued": 6, + "harmed": 364, + "missed_rescue": 0, + "avoided_harm": 492, + "selected_both_correct": 291, + "selected_both_wrong": 11 + }, + { + "budget": 0.57, + "threshold": -18.75458526611328, + "operator": "ge", + "actual_fraction": 0.5691666666666667, + "metric": 0.6783333333333333, + "rescued": 6, + "harmed": 375, + "missed_rescue": 0, + "avoided_harm": 481, + "selected_both_correct": 291, + "selected_both_wrong": 11 + }, + { + "budget": 0.58, + "threshold": -18.883766174316406, + "operator": "ge", + "actual_fraction": 0.58, + "metric": 0.67, + "rescued": 6, + "harmed": 385, + "missed_rescue": 0, + "avoided_harm": 471, + "selected_both_correct": 294, + "selected_both_wrong": 11 + }, + { + "budget": 0.59, + "threshold": -19.02402114868164, + "operator": "ge", + "actual_fraction": 0.59, + "metric": 0.66, + "rescued": 6, + "harmed": 397, + "missed_rescue": 0, + "avoided_harm": 459, + "selected_both_correct": 294, + "selected_both_wrong": 11 + }, + { + "budget": 0.6, + "threshold": -19.14906120300293, + "operator": "ge", + "actual_fraction": 0.6, + "metric": 0.6508333333333334, + "rescued": 6, + "harmed": 408, + "missed_rescue": 0, + "avoided_harm": 448, + "selected_both_correct": 295, + "selected_both_wrong": 11 + }, + { + "budget": 0.61, + "threshold": -19.287721633911133, + "operator": "ge", + "actual_fraction": 0.61, + "metric": 0.6425, + "rescued": 6, + "harmed": 418, + "missed_rescue": 0, + "avoided_harm": 438, + "selected_both_correct": 297, + "selected_both_wrong": 11 + }, + { + "budget": 0.62, + "threshold": -19.401988983154297, + "operator": "ge", + "actual_fraction": 0.62, + "metric": 0.6325, + "rescued": 6, + "harmed": 430, + "missed_rescue": 0, + "avoided_harm": 426, + "selected_both_correct": 297, + "selected_both_wrong": 11 + }, + { + "budget": 0.63, + "threshold": -19.61739158630371, + "operator": "ge", + "actual_fraction": 0.63, + "metric": 0.6233333333333333, + "rescued": 6, + "harmed": 441, + "missed_rescue": 0, + "avoided_harm": 415, + "selected_both_correct": 298, + "selected_both_wrong": 11 + }, + { + "budget": 0.64, + "threshold": -19.825759887695312, + "operator": "ge", + "actual_fraction": 0.64, + "metric": 0.615, + "rescued": 6, + "harmed": 451, + "missed_rescue": 0, + "avoided_harm": 405, + "selected_both_correct": 300, + "selected_both_wrong": 11 + }, + { + "budget": 0.65, + "threshold": -19.990713119506836, + "operator": "ge", + "actual_fraction": 0.65, + "metric": 0.6058333333333333, + "rescued": 6, + "harmed": 462, + "missed_rescue": 0, + "avoided_harm": 394, + "selected_both_correct": 301, + "selected_both_wrong": 11 + }, + { + "budget": 0.66, + "threshold": -20.097551345825195, + "operator": "ge", + "actual_fraction": 0.66, + "metric": 0.5966666666666667, + "rescued": 6, + "harmed": 473, + "missed_rescue": 0, + "avoided_harm": 383, + "selected_both_correct": 302, + "selected_both_wrong": 11 + }, + { + "budget": 0.67, + "threshold": -20.259918212890625, + "operator": "ge", + "actual_fraction": 0.67, + "metric": 0.5875, + "rescued": 6, + "harmed": 484, + "missed_rescue": 0, + "avoided_harm": 372, + "selected_both_correct": 303, + "selected_both_wrong": 11 + }, + { + "budget": 0.68, + "threshold": -20.483766555786133, + "operator": "ge", + "actual_fraction": 0.68, + "metric": 0.5783333333333334, + "rescued": 6, + "harmed": 495, + "missed_rescue": 0, + "avoided_harm": 361, + "selected_both_correct": 304, + "selected_both_wrong": 11 + }, + { + "budget": 0.69, + "threshold": -20.685535430908203, + "operator": "ge", + "actual_fraction": 0.6891666666666667, + "metric": 0.5691666666666667, + "rescued": 6, + "harmed": 506, + "missed_rescue": 0, + "avoided_harm": 350, + "selected_both_correct": 304, + "selected_both_wrong": 11 + }, + { + "budget": 0.7, + "threshold": -20.825008392333984, + "operator": "ge", + "actual_fraction": 0.7, + "metric": 0.56, + "rescued": 6, + "harmed": 517, + "missed_rescue": 0, + "avoided_harm": 339, + "selected_both_correct": 306, + "selected_both_wrong": 11 + }, + { + "budget": 0.71, + "threshold": -21.02426528930664, + "operator": "ge", + "actual_fraction": 0.71, + "metric": 0.5508333333333333, + "rescued": 6, + "harmed": 528, + "missed_rescue": 0, + "avoided_harm": 328, + "selected_both_correct": 307, + "selected_both_wrong": 11 + }, + { + "budget": 0.72, + "threshold": -21.17005157470703, + "operator": "ge", + "actual_fraction": 0.72, + "metric": 0.5425, + "rescued": 6, + "harmed": 538, + "missed_rescue": 0, + "avoided_harm": 318, + "selected_both_correct": 309, + "selected_both_wrong": 11 + }, + { + "budget": 0.73, + "threshold": -21.347705841064453, + "operator": "ge", + "actual_fraction": 0.73, + "metric": 0.535, + "rescued": 6, + "harmed": 547, + "missed_rescue": 0, + "avoided_harm": 309, + "selected_both_correct": 312, + "selected_both_wrong": 11 + }, + { + "budget": 0.74, + "threshold": -21.48891258239746, + "operator": "ge", + "actual_fraction": 0.74, + "metric": 0.525, + "rescued": 6, + "harmed": 559, + "missed_rescue": 0, + "avoided_harm": 297, + "selected_both_correct": 312, + "selected_both_wrong": 11 + }, + { + "budget": 0.75, + "threshold": -21.673301696777344, + "operator": "ge", + "actual_fraction": 0.75, + "metric": 0.5158333333333334, + "rescued": 6, + "harmed": 570, + "missed_rescue": 0, + "avoided_harm": 286, + "selected_both_correct": 313, + "selected_both_wrong": 11 + }, + { + "budget": 0.76, + "threshold": -21.752275466918945, + "operator": "ge", + "actual_fraction": 0.76, + "metric": 0.5075, + "rescued": 6, + "harmed": 580, + "missed_rescue": 0, + "avoided_harm": 276, + "selected_both_correct": 315, + "selected_both_wrong": 11 + }, + { + "budget": 0.77, + "threshold": -21.940284729003906, + "operator": "ge", + "actual_fraction": 0.77, + "metric": 0.4975, + "rescued": 6, + "harmed": 592, + "missed_rescue": 0, + "avoided_harm": 264, + "selected_both_correct": 315, + "selected_both_wrong": 11 + }, + { + "budget": 0.78, + "threshold": -22.12163543701172, + "operator": "ge", + "actual_fraction": 0.78, + "metric": 0.4875, + "rescued": 6, + "harmed": 604, + "missed_rescue": 0, + "avoided_harm": 252, + "selected_both_correct": 315, + "selected_both_wrong": 11 + }, + { + "budget": 0.79, + "threshold": -22.42583465576172, + "operator": "ge", + "actual_fraction": 0.79, + "metric": 0.4775, + "rescued": 6, + "harmed": 616, + "missed_rescue": 0, + "avoided_harm": 240, + "selected_both_correct": 315, + "selected_both_wrong": 11 + }, + { + "budget": 0.8, + "threshold": -22.678783416748047, + "operator": "ge", + "actual_fraction": 0.8, + "metric": 0.4683333333333333, + "rescued": 6, + "harmed": 627, + "missed_rescue": 0, + "avoided_harm": 229, + "selected_both_correct": 316, + "selected_both_wrong": 11 + }, + { + "budget": 0.81, + "threshold": -22.908042907714844, + "operator": "ge", + "actual_fraction": 0.81, + "metric": 0.4583333333333333, + "rescued": 6, + "harmed": 639, + "missed_rescue": 0, + "avoided_harm": 217, + "selected_both_correct": 316, + "selected_both_wrong": 11 + }, + { + "budget": 0.82, + "threshold": -23.152851104736328, + "operator": "ge", + "actual_fraction": 0.8191666666666667, + "metric": 0.4508333333333333, + "rescued": 6, + "harmed": 648, + "missed_rescue": 0, + "avoided_harm": 208, + "selected_both_correct": 318, + "selected_both_wrong": 11 + }, + { + "budget": 0.83, + "threshold": -23.55130386352539, + "operator": "ge", + "actual_fraction": 0.83, + "metric": 0.44, + "rescued": 6, + "harmed": 661, + "missed_rescue": 0, + "avoided_harm": 195, + "selected_both_correct": 318, + "selected_both_wrong": 11 + }, + { + "budget": 0.84, + "threshold": -23.888103485107422, + "operator": "ge", + "actual_fraction": 0.84, + "metric": 0.43, + "rescued": 6, + "harmed": 673, + "missed_rescue": 0, + "avoided_harm": 183, + "selected_both_correct": 318, + "selected_both_wrong": 11 + }, + { + "budget": 0.85, + "threshold": -24.166067123413086, + "operator": "ge", + "actual_fraction": 0.85, + "metric": 0.42083333333333334, + "rescued": 6, + "harmed": 684, + "missed_rescue": 0, + "avoided_harm": 172, + "selected_both_correct": 319, + "selected_both_wrong": 11 + }, + { + "budget": 0.86, + "threshold": -24.44547462463379, + "operator": "ge", + "actual_fraction": 0.86, + "metric": 0.4116666666666667, + "rescued": 6, + "harmed": 695, + "missed_rescue": 0, + "avoided_harm": 161, + "selected_both_correct": 320, + "selected_both_wrong": 11 + }, + { + "budget": 0.87, + "threshold": -24.710094451904297, + "operator": "ge", + "actual_fraction": 0.87, + "metric": 0.40166666666666667, + "rescued": 6, + "harmed": 707, + "missed_rescue": 0, + "avoided_harm": 149, + "selected_both_correct": 320, + "selected_both_wrong": 11 + }, + { + "budget": 0.88, + "threshold": -25.02822494506836, + "operator": "ge", + "actual_fraction": 0.88, + "metric": 0.3925, + "rescued": 6, + "harmed": 718, + "missed_rescue": 0, + "avoided_harm": 138, + "selected_both_correct": 321, + "selected_both_wrong": 11 + }, + { + "budget": 0.89, + "threshold": -25.415639877319336, + "operator": "ge", + "actual_fraction": 0.89, + "metric": 0.3825, + "rescued": 6, + "harmed": 730, + "missed_rescue": 0, + "avoided_harm": 126, + "selected_both_correct": 321, + "selected_both_wrong": 11 + }, + { + "budget": 0.9, + "threshold": -25.654279708862305, + "operator": "ge", + "actual_fraction": 0.9, + "metric": 0.3725, + "rescued": 6, + "harmed": 742, + "missed_rescue": 0, + "avoided_harm": 114, + "selected_both_correct": 321, + "selected_both_wrong": 11 + }, + { + "budget": 0.91, + "threshold": -25.980030059814453, + "operator": "ge", + "actual_fraction": 0.91, + "metric": 0.3641666666666667, + "rescued": 6, + "harmed": 752, + "missed_rescue": 0, + "avoided_harm": 104, + "selected_both_correct": 323, + "selected_both_wrong": 11 + }, + { + "budget": 0.92, + "threshold": -26.253726959228516, + "operator": "ge", + "actual_fraction": 0.92, + "metric": 0.3541666666666667, + "rescued": 6, + "harmed": 764, + "missed_rescue": 0, + "avoided_harm": 92, + "selected_both_correct": 323, + "selected_both_wrong": 11 + }, + { + "budget": 0.93, + "threshold": -26.68324089050293, + "operator": "ge", + "actual_fraction": 0.93, + "metric": 0.3441666666666667, + "rescued": 6, + "harmed": 776, + "missed_rescue": 0, + "avoided_harm": 80, + "selected_both_correct": 323, + "selected_both_wrong": 11 + }, + { + "budget": 0.94, + "threshold": -27.063518524169922, + "operator": "ge", + "actual_fraction": 0.94, + "metric": 0.335, + "rescued": 6, + "harmed": 787, + "missed_rescue": 0, + "avoided_harm": 69, + "selected_both_correct": 324, + "selected_both_wrong": 11 + }, + { + "budget": 0.95, + "threshold": -27.438364028930664, + "operator": "ge", + "actual_fraction": 0.95, + "metric": 0.325, + "rescued": 6, + "harmed": 799, + "missed_rescue": 0, + "avoided_harm": 57, + "selected_both_correct": 324, + "selected_both_wrong": 11 + }, + { + "budget": 0.96, + "threshold": -27.882938385009766, + "operator": "ge", + "actual_fraction": 0.96, + "metric": 0.31666666666666665, + "rescued": 6, + "harmed": 809, + "missed_rescue": 0, + "avoided_harm": 47, + "selected_both_correct": 326, + "selected_both_wrong": 11 + }, + { + "budget": 0.97, + "threshold": -28.37902069091797, + "operator": "ge", + "actual_fraction": 0.97, + "metric": 0.30666666666666664, + "rescued": 6, + "harmed": 821, + "missed_rescue": 0, + "avoided_harm": 35, + "selected_both_correct": 326, + "selected_both_wrong": 11 + }, + { + "budget": 0.98, + "threshold": -28.810405731201172, + "operator": "ge", + "actual_fraction": 0.98, + "metric": 0.2975, + "rescued": 6, + "harmed": 832, + "missed_rescue": 0, + "avoided_harm": 24, + "selected_both_correct": 327, + "selected_both_wrong": 11 + }, + { + "budget": 0.99, + "threshold": -29.864574432373047, + "operator": "ge", + "actual_fraction": 0.99, + "metric": 0.2875, + "rescued": 6, + "harmed": 844, + "missed_rescue": 0, + "avoided_harm": 12, + "selected_both_correct": 327, + "selected_both_wrong": 11 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.2775, + "rescued": 6, + "harmed": 856, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 327, + "selected_both_wrong": 11 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.81429064549534, + "rescued": 0, + "harmed": 0, + "missed_rescue": 232, + "avoided_harm": 3411, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.590226411819458, + "operator": "ge", + "actual_fraction": 0.028822920262340353, + "metric": 0.8137728684846393, + "rescued": 11, + "harmed": 14, + "missed_rescue": 221, + "avoided_harm": 3397, + "selected_both_correct": 80, + "selected_both_wrong": 62 + }, + { + "budget": 0.02, + "threshold": -1.8696393966674805, + "operator": "ge", + "actual_fraction": 0.04763548498446669, + "metric": 0.8111839834311356, + "rescued": 17, + "harmed": 35, + "missed_rescue": 215, + "avoided_harm": 3376, + "selected_both_correct": 135, + "selected_both_wrong": 89 + }, + { + "budget": 0.03, + "threshold": -2.8970229625701904, + "operator": "ge", + "actual_fraction": 0.0629962029685882, + "metric": 0.8094580600621333, + "rescued": 22, + "harmed": 50, + "missed_rescue": 210, + "avoided_harm": 3361, + "selected_both_correct": 182, + "selected_both_wrong": 111 + }, + { + "budget": 0.04, + "threshold": -3.5405211448669434, + "operator": "ge", + "actual_fraction": 0.07611322057300655, + "metric": 0.8073869520193303, + "rescued": 27, + "harmed": 67, + "missed_rescue": 205, + "avoided_harm": 3344, + "selected_both_correct": 224, + "selected_both_wrong": 123 + }, + { + "budget": 0.05, + "threshold": -4.489654064178467, + "operator": "ge", + "actual_fraction": 0.09682430100103555, + "metric": 0.8041076976182258, + "rescued": 34, + "harmed": 93, + "missed_rescue": 198, + "avoided_harm": 3318, + "selected_both_correct": 292, + "selected_both_wrong": 142 + }, + { + "budget": 0.06, + "threshold": -4.871606826782227, + "operator": "ge", + "actual_fraction": 0.10683465654124957, + "metric": 0.8025543665861236, + "rescued": 40, + "harmed": 108, + "missed_rescue": 192, + "avoided_harm": 3303, + "selected_both_correct": 315, + "selected_both_wrong": 156 + }, + { + "budget": 0.07, + "threshold": -5.445620059967041, + "operator": "ge", + "actual_fraction": 0.12374870555747325, + "metric": 0.7991025198481188, + "rescued": 49, + "harmed": 137, + "missed_rescue": 183, + "avoided_harm": 3274, + "selected_both_correct": 355, + "selected_both_wrong": 176 + }, + { + "budget": 0.08, + "threshold": -5.9812140464782715, + "operator": "ge", + "actual_fraction": 0.13876423886779427, + "metric": 0.7958232654470142, + "rescued": 56, + "harmed": 163, + "missed_rescue": 176, + "avoided_harm": 3248, + "selected_both_correct": 392, + "selected_both_wrong": 193 + }, + { + "budget": 0.09, + "threshold": -6.437536239624023, + "operator": "ge", + "actual_fraction": 0.15429754918881602, + "metric": 0.7927166033828098, + "rescued": 61, + "harmed": 186, + "missed_rescue": 171, + "avoided_harm": 3225, + "selected_both_correct": 428, + "selected_both_wrong": 219 + }, + { + "budget": 0.1, + "threshold": -6.952746868133545, + "operator": "ge", + "actual_fraction": 0.17224715222644116, + "metric": 0.789264756644805, + "rescued": 69, + "harmed": 214, + "missed_rescue": 163, + "avoided_harm": 3197, + "selected_both_correct": 468, + "selected_both_wrong": 247 + }, + { + "budget": 0.11, + "threshold": -7.571210861206055, + "operator": "ge", + "actual_fraction": 0.20089748015188125, + "metric": 0.7830514325163963, + "rescued": 78, + "harmed": 259, + "missed_rescue": 154, + "avoided_harm": 3152, + "selected_both_correct": 535, + "selected_both_wrong": 292 + }, + { + "budget": 0.12, + "threshold": -7.767908573150635, + "operator": "ge", + "actual_fraction": 0.21108042802899551, + "metric": 0.7797721781152916, + "rescued": 81, + "harmed": 281, + "missed_rescue": 151, + "avoided_harm": 3130, + "selected_both_correct": 556, + "selected_both_wrong": 305 + }, + { + "budget": 0.13, + "threshold": -8.024438858032227, + "operator": "ge", + "actual_fraction": 0.22195374525371073, + "metric": 0.777183293061788, + "rescued": 83, + "harmed": 298, + "missed_rescue": 149, + "avoided_harm": 3113, + "selected_both_correct": 585, + "selected_both_wrong": 320 + }, + { + "budget": 0.14, + "threshold": -8.368799209594727, + "operator": "ge", + "actual_fraction": 0.2359337245426303, + "metric": 0.7732136693130826, + "rescued": 85, + "harmed": 323, + "missed_rescue": 147, + "avoided_harm": 3088, + "selected_both_correct": 616, + "selected_both_wrong": 343 + }, + { + "budget": 0.15, + "threshold": -8.57835578918457, + "operator": "ge", + "actual_fraction": 0.24508111839834312, + "metric": 0.7711425612702796, + "rescued": 90, + "harmed": 340, + "missed_rescue": 142, + "avoided_harm": 3071, + "selected_both_correct": 634, + "selected_both_wrong": 356 + }, + { + "budget": 0.16, + "threshold": -8.983278274536133, + "operator": "ge", + "actual_fraction": 0.2587159130134622, + "metric": 0.7659647911632723, + "rescued": 93, + "harmed": 373, + "missed_rescue": 139, + "avoided_harm": 3038, + "selected_both_correct": 664, + "selected_both_wrong": 369 + }, + { + "budget": 0.17, + "threshold": -9.323188781738281, + "operator": "ge", + "actual_fraction": 0.27217811529168107, + "metric": 0.7613047980669658, + "rescued": 96, + "harmed": 403, + "missed_rescue": 136, + "avoided_harm": 3008, + "selected_both_correct": 691, + "selected_both_wrong": 387 + }, + { + "budget": 0.18, + "threshold": -9.623194694519043, + "operator": "ge", + "actual_fraction": 0.2832240248532965, + "metric": 0.7594062823610632, + "rescued": 100, + "harmed": 418, + "missed_rescue": 132, + "avoided_harm": 2993, + "selected_both_correct": 722, + "selected_both_wrong": 401 + }, + { + "budget": 0.19, + "threshold": -9.886858940124512, + "operator": "ge", + "actual_fraction": 0.2927166033828098, + "metric": 0.7566448049706593, + "rescued": 104, + "harmed": 438, + "missed_rescue": 128, + "avoided_harm": 2973, + "selected_both_correct": 743, + "selected_both_wrong": 411 + }, + { + "budget": 0.2, + "threshold": -10.168641090393066, + "operator": "ge", + "actual_fraction": 0.304280289955126, + "metric": 0.7509492578529513, + "rescued": 106, + "harmed": 473, + "missed_rescue": 126, + "avoided_harm": 2938, + "selected_both_correct": 765, + "selected_both_wrong": 419 + }, + { + "budget": 0.21, + "threshold": -10.403701782226562, + "operator": "ge", + "actual_fraction": 0.31308249913703834, + "metric": 0.7474974111149465, + "rescued": 107, + "harmed": 494, + "missed_rescue": 125, + "avoided_harm": 2917, + "selected_both_correct": 788, + "selected_both_wrong": 425 + }, + { + "budget": 0.22, + "threshold": -10.783614158630371, + "operator": "ge", + "actual_fraction": 0.32827062478425956, + "metric": 0.744218156713842, + "rescued": 115, + "harmed": 521, + "missed_rescue": 117, + "avoided_harm": 2890, + "selected_both_correct": 824, + "selected_both_wrong": 442 + }, + { + "budget": 0.23, + "threshold": -11.247787475585938, + "operator": "ge", + "actual_fraction": 0.351397997928892, + "metric": 0.7343803935105281, + "rescued": 122, + "harmed": 585, + "missed_rescue": 110, + "avoided_harm": 2826, + "selected_both_correct": 863, + "selected_both_wrong": 466 + }, + { + "budget": 0.24, + "threshold": -11.575923919677734, + "operator": "ge", + "actual_fraction": 0.3648602002071108, + "metric": 0.7290300310666207, + "rescued": 129, + "harmed": 623, + "missed_rescue": 103, + "avoided_harm": 2788, + "selected_both_correct": 884, + "selected_both_wrong": 478 + }, + { + "budget": 0.25, + "threshold": -12.19352912902832, + "operator": "ge", + "actual_fraction": 0.39299275112185017, + "metric": 0.7179841215050051, + "rescued": 139, + "harmed": 697, + "missed_rescue": 93, + "avoided_harm": 2714, + "selected_both_correct": 929, + "selected_both_wrong": 512 + }, + { + "budget": 0.26, + "threshold": -12.560779571533203, + "operator": "ge", + "actual_fraction": 0.412150500517777, + "metric": 0.7096996893337936, + "rescued": 142, + "harmed": 748, + "missed_rescue": 90, + "avoided_harm": 2663, + "selected_both_correct": 955, + "selected_both_wrong": 543 + }, + { + "budget": 0.27, + "threshold": -12.845895767211914, + "operator": "ge", + "actual_fraction": 0.42716603382809804, + "metric": 0.7019330341732827, + "rescued": 148, + "harmed": 799, + "missed_rescue": 84, + "avoided_harm": 2612, + "selected_both_correct": 974, + "selected_both_wrong": 554 + }, + { + "budget": 0.28, + "threshold": -13.137770652770996, + "operator": "ge", + "actual_fraction": 0.4432171211598205, + "metric": 0.6929582326544701, + "rescued": 155, + "harmed": 858, + "missed_rescue": 77, + "avoided_harm": 2553, + "selected_both_correct": 988, + "selected_both_wrong": 567 + }, + { + "budget": 0.29, + "threshold": -13.375581741333008, + "operator": "ge", + "actual_fraction": 0.454263030721436, + "metric": 0.687780462547463, + "rescued": 158, + "harmed": 891, + "missed_rescue": 74, + "avoided_harm": 2520, + "selected_both_correct": 1007, + "selected_both_wrong": 576 + }, + { + "budget": 0.3, + "threshold": -13.533597946166992, + "operator": "ge", + "actual_fraction": 0.46047635484984467, + "metric": 0.6846738004832585, + "rescued": 159, + "harmed": 910, + "missed_rescue": 73, + "avoided_harm": 2501, + "selected_both_correct": 1016, + "selected_both_wrong": 583 + }, + { + "budget": 0.31, + "threshold": -13.760375022888184, + "operator": "ge", + "actual_fraction": 0.4722126337590611, + "metric": 0.6781152916810493, + "rescued": 161, + "harmed": 950, + "missed_rescue": 71, + "avoided_harm": 2461, + "selected_both_correct": 1031, + "selected_both_wrong": 594 + }, + { + "budget": 0.32, + "threshold": -13.987640380859375, + "operator": "ge", + "actual_fraction": 0.4827407663099758, + "metric": 0.6732827062478426, + "rescued": 163, + "harmed": 980, + "missed_rescue": 69, + "avoided_harm": 2431, + "selected_both_correct": 1050, + "selected_both_wrong": 604 + }, + { + "budget": 0.33, + "threshold": -14.21721076965332, + "operator": "ge", + "actual_fraction": 0.4929237141870901, + "metric": 0.6677597514670348, + "rescued": 167, + "harmed": 1016, + "missed_rescue": 65, + "avoided_harm": 2395, + "selected_both_correct": 1064, + "selected_both_wrong": 609 + }, + { + "budget": 0.34, + "threshold": -14.470904350280762, + "operator": "ge", + "actual_fraction": 0.5039696237487056, + "metric": 0.6617190196755264, + "rescued": 169, + "harmed": 1053, + "missed_rescue": 63, + "avoided_harm": 2358, + "selected_both_correct": 1076, + "selected_both_wrong": 622 + }, + { + "budget": 0.35, + "threshold": -14.772564888000488, + "operator": "ge", + "actual_fraction": 0.5212288574387297, + "metric": 0.6518812564722126, + "rescued": 176, + "harmed": 1117, + "missed_rescue": 56, + "avoided_harm": 2294, + "selected_both_correct": 1092, + "selected_both_wrong": 635 + }, + { + "budget": 0.36, + "threshold": -15.04111099243164, + "operator": "ge", + "actual_fraction": 0.5369347600966518, + "metric": 0.6425612702795996, + "rescued": 178, + "harmed": 1173, + "missed_rescue": 54, + "avoided_harm": 2238, + "selected_both_correct": 1111, + "selected_both_wrong": 649 + }, + { + "budget": 0.37, + "threshold": -15.203546524047852, + "operator": "ge", + "actual_fraction": 0.5452191922678633, + "metric": 0.6384190541939938, + "rescued": 184, + "harmed": 1203, + "missed_rescue": 48, + "avoided_harm": 2208, + "selected_both_correct": 1118, + "selected_both_wrong": 654 + }, + { + "budget": 0.38, + "threshold": -15.385844230651855, + "operator": "ge", + "actual_fraction": 0.5545391784604764, + "metric": 0.6325509147393855, + "rescued": 186, + "harmed": 1239, + "missed_rescue": 46, + "avoided_harm": 2172, + "selected_both_correct": 1127, + "selected_both_wrong": 661 + }, + { + "budget": 0.39, + "threshold": -15.582735061645508, + "operator": "ge", + "actual_fraction": 0.5626510182947877, + "metric": 0.6263375906109768, + "rescued": 186, + "harmed": 1275, + "missed_rescue": 46, + "avoided_harm": 2136, + "selected_both_correct": 1133, + "selected_both_wrong": 666 + }, + { + "budget": 0.4, + "threshold": -15.746196746826172, + "operator": "ge", + "actual_fraction": 0.5698998964445978, + "metric": 0.6221953745253711, + "rescued": 190, + "harmed": 1303, + "missed_rescue": 42, + "avoided_harm": 2108, + "selected_both_correct": 1138, + "selected_both_wrong": 671 + }, + { + "budget": 0.41, + "threshold": -15.894475936889648, + "operator": "ge", + "actual_fraction": 0.576458405246807, + "metric": 0.6189161201242664, + "rescued": 194, + "harmed": 1326, + "missed_rescue": 38, + "avoided_harm": 2085, + "selected_both_correct": 1144, + "selected_both_wrong": 676 + }, + { + "budget": 0.42, + "threshold": -16.114011764526367, + "operator": "ge", + "actual_fraction": 0.5868139454608216, + "metric": 0.6120124266482568, + "rescued": 194, + "harmed": 1366, + "missed_rescue": 38, + "avoided_harm": 2045, + "selected_both_correct": 1156, + "selected_both_wrong": 684 + }, + { + "budget": 0.43, + "threshold": -16.257381439208984, + "operator": "ge", + "actual_fraction": 0.5916465308940283, + "metric": 0.6090783569209527, + "rescued": 196, + "harmed": 1385, + "missed_rescue": 36, + "avoided_harm": 2026, + "selected_both_correct": 1159, + "selected_both_wrong": 688 + }, + { + "budget": 0.44, + "threshold": -16.480209350585938, + "operator": "ge", + "actual_fraction": 0.6016568864342423, + "metric": 0.6026924404556437, + "rescued": 198, + "harmed": 1424, + "missed_rescue": 34, + "avoided_harm": 1987, + "selected_both_correct": 1168, + "selected_both_wrong": 696 + }, + { + "budget": 0.45, + "threshold": -16.728397369384766, + "operator": "ge", + "actual_fraction": 0.6140835346910597, + "metric": 0.5931998619261305, + "rescued": 200, + "harmed": 1481, + "missed_rescue": 32, + "avoided_harm": 1930, + "selected_both_correct": 1173, + "selected_both_wrong": 704 + }, + { + "budget": 0.46, + "threshold": -16.844228744506836, + "operator": "ge", + "actual_fraction": 0.6180531584397653, + "metric": 0.5906109768726269, + "rescued": 201, + "harmed": 1497, + "missed_rescue": 31, + "avoided_harm": 1914, + "selected_both_correct": 1177, + "selected_both_wrong": 706 + }, + { + "budget": 0.47, + "threshold": -17.066635131835938, + "operator": "ge", + "actual_fraction": 0.6273731446323784, + "metric": 0.5845702450811184, + "rescued": 204, + "harmed": 1535, + "missed_rescue": 28, + "avoided_harm": 1876, + "selected_both_correct": 1185, + "selected_both_wrong": 711 + }, + { + "budget": 0.48, + "threshold": -17.25113296508789, + "operator": "ge", + "actual_fraction": 0.6360027614773904, + "metric": 0.577839143942009, + "rescued": 205, + "harmed": 1575, + "missed_rescue": 27, + "avoided_harm": 1836, + "selected_both_correct": 1188, + "selected_both_wrong": 717 + }, + { + "budget": 0.49, + "threshold": -17.416290283203125, + "operator": "ge", + "actual_fraction": 0.6429064549534, + "metric": 0.572833966171902, + "rescued": 206, + "harmed": 1605, + "missed_rescue": 26, + "avoided_harm": 1806, + "selected_both_correct": 1189, + "selected_both_wrong": 725 + }, + { + "budget": 0.5, + "threshold": -17.648540496826172, + "operator": "ge", + "actual_fraction": 0.6534345875043148, + "metric": 0.5645495340006904, + "rescued": 207, + "harmed": 1654, + "missed_rescue": 25, + "avoided_harm": 1757, + "selected_both_correct": 1195, + "selected_both_wrong": 730 + }, + { + "budget": 0.51, + "threshold": -17.833698272705078, + "operator": "ge", + "actual_fraction": 0.661373835001726, + "metric": 0.5586813945460821, + "rescued": 207, + "harmed": 1688, + "missed_rescue": 25, + "avoided_harm": 1723, + "selected_both_correct": 1201, + "selected_both_wrong": 736 + }, + { + "budget": 0.52, + "threshold": -17.978940963745117, + "operator": "ge", + "actual_fraction": 0.6694856748360373, + "metric": 0.5524680704176734, + "rescued": 207, + "harmed": 1724, + "missed_rescue": 25, + "avoided_harm": 1687, + "selected_both_correct": 1208, + "selected_both_wrong": 740 + }, + { + "budget": 0.53, + "threshold": -18.172624588012695, + "operator": "ge", + "actual_fraction": 0.6789782533655506, + "metric": 0.5440110459095616, + "rescued": 207, + "harmed": 1773, + "missed_rescue": 25, + "avoided_harm": 1638, + "selected_both_correct": 1210, + "selected_both_wrong": 744 + }, + { + "budget": 0.54, + "threshold": -18.375865936279297, + "operator": "ge", + "actual_fraction": 0.6888160165688644, + "metric": 0.5365895754228512, + "rescued": 207, + "harmed": 1816, + "missed_rescue": 25, + "avoided_harm": 1595, + "selected_both_correct": 1215, + "selected_both_wrong": 753 + }, + { + "budget": 0.55, + "threshold": -18.543420791625977, + "operator": "ge", + "actual_fraction": 0.6958923023817742, + "metric": 0.5308940283051432, + "rescued": 208, + "harmed": 1850, + "missed_rescue": 24, + "avoided_harm": 1561, + "selected_both_correct": 1216, + "selected_both_wrong": 758 + }, + { + "budget": 0.56, + "threshold": -18.668851852416992, + "operator": "ge", + "actual_fraction": 0.7019330341732827, + "metric": 0.5269244045564377, + "rescued": 209, + "harmed": 1874, + "missed_rescue": 23, + "avoided_harm": 1537, + "selected_both_correct": 1223, + "selected_both_wrong": 761 + }, + { + "budget": 0.57, + "threshold": -18.75458526611328, + "operator": "ge", + "actual_fraction": 0.705730065585088, + "metric": 0.5234725578184328, + "rescued": 209, + "harmed": 1894, + "missed_rescue": 23, + "avoided_harm": 1517, + "selected_both_correct": 1224, + "selected_both_wrong": 762 + }, + { + "budget": 0.58, + "threshold": -18.883766174316406, + "operator": "ge", + "actual_fraction": 0.7129789437348981, + "metric": 0.5172592336900241, + "rescued": 210, + "harmed": 1931, + "missed_rescue": 22, + "avoided_harm": 1480, + "selected_both_correct": 1227, + "selected_both_wrong": 763 + }, + { + "budget": 0.59, + "threshold": -19.02402114868164, + "operator": "ge", + "actual_fraction": 0.7198826372109078, + "metric": 0.5110459095616154, + "rescued": 211, + "harmed": 1968, + "missed_rescue": 21, + "avoided_harm": 1443, + "selected_both_correct": 1229, + "selected_both_wrong": 763 + }, + { + "budget": 0.6, + "threshold": -19.14906120300293, + "operator": "ge", + "actual_fraction": 0.7255781843286158, + "metric": 0.506385916465309, + "rescued": 213, + "harmed": 1997, + "missed_rescue": 19, + "avoided_harm": 1414, + "selected_both_correct": 1231, + "selected_both_wrong": 763 + }, + { + "budget": 0.61, + "threshold": -19.287721633911133, + "operator": "ge", + "actual_fraction": 0.7323092854677252, + "metric": 0.5008629616845012, + "rescued": 214, + "harmed": 2030, + "missed_rescue": 18, + "avoided_harm": 1381, + "selected_both_correct": 1234, + "selected_both_wrong": 765 + }, + { + "budget": 0.62, + "threshold": -19.401988983154297, + "operator": "ge", + "actual_fraction": 0.7383500172592337, + "metric": 0.49568519157749397, + "rescued": 214, + "harmed": 2060, + "missed_rescue": 18, + "avoided_harm": 1351, + "selected_both_correct": 1237, + "selected_both_wrong": 767 + }, + { + "budget": 0.63, + "threshold": -19.61739158630371, + "operator": "ge", + "actual_fraction": 0.7466344494304453, + "metric": 0.48895409043838456, + "rescued": 215, + "harmed": 2100, + "missed_rescue": 17, + "avoided_harm": 1311, + "selected_both_correct": 1240, + "selected_both_wrong": 771 + }, + { + "budget": 0.64, + "threshold": -19.825759887695312, + "operator": "ge", + "actual_fraction": 0.7552640662754574, + "metric": 0.4820503969623749, + "rescued": 217, + "harmed": 2142, + "missed_rescue": 15, + "avoided_harm": 1269, + "selected_both_correct": 1241, + "selected_both_wrong": 776 + }, + { + "budget": 0.65, + "threshold": -19.990713119506836, + "operator": "ge", + "actual_fraction": 0.7600966517086641, + "metric": 0.4777355885398688, + "rescued": 217, + "harmed": 2167, + "missed_rescue": 15, + "avoided_harm": 1244, + "selected_both_correct": 1243, + "selected_both_wrong": 777 + }, + { + "budget": 0.66, + "threshold": -20.097551345825195, + "operator": "ge", + "actual_fraction": 0.7645840524680704, + "metric": 0.4737659647911633, + "rescued": 217, + "harmed": 2190, + "missed_rescue": 15, + "avoided_harm": 1221, + "selected_both_correct": 1246, + "selected_both_wrong": 777 + }, + { + "budget": 0.67, + "threshold": -20.259918212890625, + "operator": "ge", + "actual_fraction": 0.7697618225750776, + "metric": 0.46996893337935797, + "rescued": 219, + "harmed": 2214, + "missed_rescue": 13, + "avoided_harm": 1197, + "selected_both_correct": 1248, + "selected_both_wrong": 779 + }, + { + "budget": 0.68, + "threshold": -20.483766555786133, + "operator": "ge", + "actual_fraction": 0.7797721781152916, + "metric": 0.4630652399033483, + "rescued": 221, + "harmed": 2256, + "missed_rescue": 11, + "avoided_harm": 1155, + "selected_both_correct": 1254, + "selected_both_wrong": 787 + }, + { + "budget": 0.69, + "threshold": -20.685535430908203, + "operator": "ge", + "actual_fraction": 0.7904729030031067, + "metric": 0.45460821539523644, + "rescued": 222, + "harmed": 2306, + "missed_rescue": 10, + "avoided_harm": 1105, + "selected_both_correct": 1260, + "selected_both_wrong": 792 + }, + { + "budget": 0.7, + "threshold": -20.825008392333984, + "operator": "ge", + "actual_fraction": 0.7963410424577149, + "metric": 0.4490852606144287, + "rescued": 222, + "harmed": 2338, + "missed_rescue": 10, + "avoided_harm": 1073, + "selected_both_correct": 1260, + "selected_both_wrong": 794 + }, + { + "budget": 0.71, + "threshold": -21.02426528930664, + "operator": "ge", + "actual_fraction": 0.8065239903348291, + "metric": 0.4406282361063169, + "rescued": 222, + "harmed": 2387, + "missed_rescue": 10, + "avoided_harm": 1024, + "selected_both_correct": 1266, + "selected_both_wrong": 798 + }, + { + "budget": 0.72, + "threshold": -21.17005157470703, + "operator": "ge", + "actual_fraction": 0.8129099068001381, + "metric": 0.43596824301001036, + "rescued": 223, + "harmed": 2415, + "missed_rescue": 9, + "avoided_harm": 996, + "selected_both_correct": 1270, + "selected_both_wrong": 802 + }, + { + "budget": 0.73, + "threshold": -21.347705841064453, + "operator": "ge", + "actual_fraction": 0.8194684156023473, + "metric": 0.43061788056610284, + "rescued": 225, + "harmed": 2448, + "missed_rescue": 7, + "avoided_harm": 963, + "selected_both_correct": 1271, + "selected_both_wrong": 804 + }, + { + "budget": 0.74, + "threshold": -21.48891258239746, + "operator": "ge", + "actual_fraction": 0.8244735933724543, + "metric": 0.4261304798066966, + "rescued": 225, + "harmed": 2474, + "missed_rescue": 7, + "avoided_harm": 937, + "selected_both_correct": 1273, + "selected_both_wrong": 805 + }, + { + "budget": 0.75, + "threshold": -21.673301696777344, + "operator": "ge", + "actual_fraction": 0.8301691404901622, + "metric": 0.4211253020365896, + "rescued": 225, + "harmed": 2503, + "missed_rescue": 7, + "avoided_harm": 908, + "selected_both_correct": 1276, + "selected_both_wrong": 806 + }, + { + "budget": 0.76, + "threshold": -21.752275466918945, + "operator": "ge", + "actual_fraction": 0.8332758025543666, + "metric": 0.41870900931998617, + "rescued": 225, + "harmed": 2517, + "missed_rescue": 7, + "avoided_harm": 894, + "selected_both_correct": 1278, + "selected_both_wrong": 808 + }, + { + "budget": 0.77, + "threshold": -21.940284729003906, + "operator": "ge", + "actual_fraction": 0.8413876423886779, + "metric": 0.41146013117017605, + "rescued": 226, + "harmed": 2560, + "missed_rescue": 6, + "avoided_harm": 851, + "selected_both_correct": 1280, + "selected_both_wrong": 809 + }, + { + "budget": 0.78, + "threshold": -22.12163543701172, + "operator": "ge", + "actual_fraction": 0.847083189506386, + "metric": 0.40645495340006904, + "rescued": 226, + "harmed": 2589, + "missed_rescue": 6, + "avoided_harm": 822, + "selected_both_correct": 1283, + "selected_both_wrong": 810 + }, + { + "budget": 0.79, + "threshold": -22.42583465576172, + "operator": "ge", + "actual_fraction": 0.855712806351398, + "metric": 0.39851570590265795, + "rescued": 226, + "harmed": 2635, + "missed_rescue": 6, + "avoided_harm": 776, + "selected_both_correct": 1284, + "selected_both_wrong": 813 + }, + { + "budget": 0.8, + "threshold": -22.678783416748047, + "operator": "ge", + "actual_fraction": 0.8648602002071109, + "metric": 0.3909216430790473, + "rescued": 227, + "harmed": 2680, + "missed_rescue": 5, + "avoided_harm": 731, + "selected_both_correct": 1286, + "selected_both_wrong": 818 + }, + { + "budget": 0.81, + "threshold": -22.908042907714844, + "operator": "ge", + "actual_fraction": 0.8724542630307215, + "metric": 0.38419054193993785, + "rescued": 227, + "harmed": 2719, + "missed_rescue": 5, + "avoided_harm": 692, + "selected_both_correct": 1289, + "selected_both_wrong": 820 + }, + { + "budget": 0.82, + "threshold": -23.152851104736328, + "operator": "ge", + "actual_fraction": 0.8788401794960303, + "metric": 0.3786675871591301, + "rescued": 227, + "harmed": 2751, + "missed_rescue": 5, + "avoided_harm": 660, + "selected_both_correct": 1292, + "selected_both_wrong": 822 + }, + { + "budget": 0.83, + "threshold": -23.55130386352539, + "operator": "ge", + "actual_fraction": 0.8910942354159476, + "metric": 0.36727649292371417, + "rescued": 227, + "harmed": 2817, + "missed_rescue": 5, + "avoided_harm": 594, + "selected_both_correct": 1294, + "selected_both_wrong": 825 + }, + { + "budget": 0.84, + "threshold": -23.888103485107422, + "operator": "ge", + "actual_fraction": 0.8995512599240594, + "metric": 0.3598550224370038, + "rescued": 228, + "harmed": 2861, + "missed_rescue": 4, + "avoided_harm": 550, + "selected_both_correct": 1296, + "selected_both_wrong": 827 + }, + { + "budget": 0.85, + "threshold": -24.166067123413086, + "operator": "ge", + "actual_fraction": 0.9111149464963756, + "metric": 0.3486365205384881, + "rescued": 228, + "harmed": 2926, + "missed_rescue": 4, + "avoided_harm": 485, + "selected_both_correct": 1296, + "selected_both_wrong": 829 + }, + { + "budget": 0.86, + "threshold": -24.44547462463379, + "operator": "ge", + "actual_fraction": 0.9173282706247843, + "metric": 0.34276838108387986, + "rescued": 228, + "harmed": 2960, + "missed_rescue": 4, + "avoided_harm": 451, + "selected_both_correct": 1297, + "selected_both_wrong": 830 + }, + { + "budget": 0.87, + "threshold": -24.710094451904297, + "operator": "ge", + "actual_fraction": 0.9252675181221953, + "metric": 0.33569209527097, + "rescued": 229, + "harmed": 3002, + "missed_rescue": 3, + "avoided_harm": 409, + "selected_both_correct": 1298, + "selected_both_wrong": 832 + }, + { + "budget": 0.88, + "threshold": -25.02822494506836, + "operator": "ge", + "actual_fraction": 0.9313082499137039, + "metric": 0.32982395581636176, + "rescued": 229, + "harmed": 3036, + "missed_rescue": 3, + "avoided_harm": 375, + "selected_both_correct": 1299, + "selected_both_wrong": 832 + }, + { + "budget": 0.89, + "threshold": -25.415639877319336, + "operator": "ge", + "actual_fraction": 0.937866758715913, + "metric": 0.32361063168795307, + "rescued": 229, + "harmed": 3072, + "missed_rescue": 3, + "avoided_harm": 339, + "selected_both_correct": 1300, + "selected_both_wrong": 833 + }, + { + "budget": 0.9, + "threshold": -25.654279708862305, + "operator": "ge", + "actual_fraction": 0.9414911977908181, + "metric": 0.32033137728684846, + "rescued": 229, + "harmed": 3091, + "missed_rescue": 3, + "avoided_harm": 320, + "selected_both_correct": 1301, + "selected_both_wrong": 834 + }, + { + "budget": 0.91, + "threshold": -25.980030059814453, + "operator": "ge", + "actual_fraction": 0.9471867449085261, + "metric": 0.3151536071798412, + "rescued": 230, + "harmed": 3122, + "missed_rescue": 2, + "avoided_harm": 289, + "selected_both_correct": 1301, + "selected_both_wrong": 835 + }, + { + "budget": 0.92, + "threshold": -26.253726959228516, + "operator": "ge", + "actual_fraction": 0.9528822920262341, + "metric": 0.309975837072834, + "rescued": 230, + "harmed": 3152, + "missed_rescue": 2, + "avoided_harm": 259, + "selected_both_correct": 1302, + "selected_both_wrong": 837 + }, + { + "budget": 0.93, + "threshold": -26.68324089050293, + "operator": "ge", + "actual_fraction": 0.9589230238177425, + "metric": 0.304280289955126, + "rescued": 230, + "harmed": 3185, + "missed_rescue": 2, + "avoided_harm": 226, + "selected_both_correct": 1303, + "selected_both_wrong": 838 + }, + { + "budget": 0.94, + "threshold": -27.063518524169922, + "operator": "ge", + "actual_fraction": 0.9646185709354504, + "metric": 0.2989299275112185, + "rescued": 230, + "harmed": 3216, + "missed_rescue": 2, + "avoided_harm": 195, + "selected_both_correct": 1304, + "selected_both_wrong": 839 + }, + { + "budget": 0.95, + "threshold": -27.438364028930664, + "operator": "ge", + "actual_fraction": 0.968588194684156, + "metric": 0.2956506731101139, + "rescued": 230, + "harmed": 3235, + "missed_rescue": 2, + "avoided_harm": 176, + "selected_both_correct": 1306, + "selected_both_wrong": 841 + }, + { + "budget": 0.96, + "threshold": -27.882938385009766, + "operator": "ge", + "actual_fraction": 0.9744563341387642, + "metric": 0.2897825336555057, + "rescued": 230, + "harmed": 3269, + "missed_rescue": 2, + "avoided_harm": 142, + "selected_both_correct": 1306, + "selected_both_wrong": 841 + }, + { + "budget": 0.97, + "threshold": -28.37902069091797, + "operator": "ge", + "actual_fraction": 0.979288919571971, + "metric": 0.2849499482222989, + "rescued": 230, + "harmed": 3297, + "missed_rescue": 2, + "avoided_harm": 114, + "selected_both_correct": 1306, + "selected_both_wrong": 841 + }, + { + "budget": 0.98, + "threshold": -28.810405731201172, + "operator": "ge", + "actual_fraction": 0.9837763203313773, + "metric": 0.2804625474628927, + "rescued": 230, + "harmed": 3323, + "missed_rescue": 2, + "avoided_harm": 88, + "selected_both_correct": 1306, + "selected_both_wrong": 841 + }, + { + "budget": 0.99, + "threshold": -29.864574432373047, + "operator": "ge", + "actual_fraction": 0.9917155678287884, + "metric": 0.27338626164998275, + "rescued": 231, + "harmed": 3365, + "missed_rescue": 1, + "avoided_harm": 46, + "selected_both_correct": 1307, + "selected_both_wrong": 843 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.26561960648947186, + "rescued": 232, + "harmed": 3411, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 1307, + "selected_both_wrong": 844 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 327, + "rescuable": 6, + "harmful": 856, + "both_wrong": 11, + "unit": "sample", + "total": 1200 + }, + "test": { + "both_correct": 1307, + "rescuable": 232, + "harmful": 3411, + "both_wrong": 844, + "unit": "sample", + "total": 5794 + } + }, + "small": { + "validation": 0.9858333333333333, + "test": 0.81429064549534 + }, + "large": { + "validation": 0.2775, + "test": 0.26561960648947186 + }, + "learned_minus_confidence_test": 0.014497756299620246, + "learned_minus_small_test": -0.0005177770107007706, + "learned_minus_large_test": 0.5481532619951675 + }, + { + "key": [ + "cub", + "cub", + "Qwen3.8" + ], + "expert_display": "ResNet", + "run": "shared_12_cub_Qwen3.8", + "validation_rows": 1200, + "test_rows": 5794, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/cub/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 1, + "validation_area_0_25": 0.9171666666666667, + "validation_point": { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9825, + "rescued": 1, + "harmed": 5, + "missed_rescue": 1, + "avoided_harm": 519, + "selected_both_correct": 1, + "selected_both_wrong": 5 + }, + "test_point": { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.113565757680359, + "metric": 0.8158439765274422, + "rescued": 132, + "harmed": 123, + "missed_rescue": 233, + "avoided_harm": 1784, + "selected_both_correct": 108, + "selected_both_wrong": 295 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9858333333333333, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2, + "avoided_harm": 524, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9825, + "rescued": 1, + "harmed": 5, + "missed_rescue": 1, + "avoided_harm": 519, + "selected_both_correct": 1, + "selected_both_wrong": 5 + }, + { + "budget": 0.02, + "threshold": -0.3627302646636963, + "operator": "ge", + "actual_fraction": 0.02, + "metric": 0.9775, + "rescued": 1, + "harmed": 11, + "missed_rescue": 1, + "avoided_harm": 513, + "selected_both_correct": 3, + "selected_both_wrong": 9 + }, + { + "budget": 0.03, + "threshold": -0.4222676455974579, + "operator": "ge", + "actual_fraction": 0.03, + "metric": 0.9725, + "rescued": 2, + "harmed": 18, + "missed_rescue": 0, + "avoided_harm": 506, + "selected_both_correct": 6, + "selected_both_wrong": 10 + }, + { + "budget": 0.04, + "threshold": -0.46618229150772095, + "operator": "ge", + "actual_fraction": 0.04, + "metric": 0.9675, + "rescued": 2, + "harmed": 24, + "missed_rescue": 0, + "avoided_harm": 500, + "selected_both_correct": 10, + "selected_both_wrong": 12 + }, + { + "budget": 0.05, + "threshold": -0.5066347122192383, + "operator": "ge", + "actual_fraction": 0.05, + "metric": 0.9608333333333333, + "rescued": 2, + "harmed": 32, + "missed_rescue": 0, + "avoided_harm": 492, + "selected_both_correct": 14, + "selected_both_wrong": 12 + }, + { + "budget": 0.06, + "threshold": -0.5393391251564026, + "operator": "ge", + "actual_fraction": 0.06, + "metric": 0.955, + "rescued": 2, + "harmed": 39, + "missed_rescue": 0, + "avoided_harm": 485, + "selected_both_correct": 17, + "selected_both_wrong": 14 + }, + { + "budget": 0.07, + "threshold": -0.5679479837417603, + "operator": "ge", + "actual_fraction": 0.07, + "metric": 0.9508333333333333, + "rescued": 2, + "harmed": 44, + "missed_rescue": 0, + "avoided_harm": 480, + "selected_both_correct": 23, + "selected_both_wrong": 15 + }, + { + "budget": 0.08, + "threshold": -0.5907968282699585, + "operator": "ge", + "actual_fraction": 0.08, + "metric": 0.945, + "rescued": 2, + "harmed": 51, + "missed_rescue": 0, + "avoided_harm": 473, + "selected_both_correct": 28, + "selected_both_wrong": 15 + }, + { + "budget": 0.09, + "threshold": -0.6168767809867859, + "operator": "ge", + "actual_fraction": 0.09, + "metric": 0.9391666666666667, + "rescued": 2, + "harmed": 58, + "missed_rescue": 0, + "avoided_harm": 466, + "selected_both_correct": 33, + "selected_both_wrong": 15 + }, + { + "budget": 0.1, + "threshold": -0.6361178159713745, + "operator": "ge", + "actual_fraction": 0.1, + "metric": 0.9333333333333333, + "rescued": 2, + "harmed": 65, + "missed_rescue": 0, + "avoided_harm": 459, + "selected_both_correct": 38, + "selected_both_wrong": 15 + }, + { + "budget": 0.11, + "threshold": -0.6547113656997681, + "operator": "ge", + "actual_fraction": 0.11, + "metric": 0.9291666666666667, + "rescued": 2, + "harmed": 70, + "missed_rescue": 0, + "avoided_harm": 454, + "selected_both_correct": 45, + "selected_both_wrong": 15 + }, + { + "budget": 0.12, + "threshold": -0.6680066585540771, + "operator": "ge", + "actual_fraction": 0.12, + "metric": 0.9225, + "rescued": 2, + "harmed": 78, + "missed_rescue": 0, + "avoided_harm": 446, + "selected_both_correct": 49, + "selected_both_wrong": 15 + }, + { + "budget": 0.13, + "threshold": -0.6809748411178589, + "operator": "ge", + "actual_fraction": 0.13, + "metric": 0.9141666666666667, + "rescued": 2, + "harmed": 88, + "missed_rescue": 0, + "avoided_harm": 436, + "selected_both_correct": 51, + "selected_both_wrong": 15 + }, + { + "budget": 0.14, + "threshold": -0.6963143348693848, + "operator": "ge", + "actual_fraction": 0.14, + "metric": 0.9083333333333333, + "rescued": 2, + "harmed": 95, + "missed_rescue": 0, + "avoided_harm": 429, + "selected_both_correct": 56, + "selected_both_wrong": 15 + }, + { + "budget": 0.15, + "threshold": -0.7065573930740356, + "operator": "ge", + "actual_fraction": 0.15, + "metric": 0.9033333333333333, + "rescued": 2, + "harmed": 101, + "missed_rescue": 0, + "avoided_harm": 423, + "selected_both_correct": 62, + "selected_both_wrong": 15 + }, + { + "budget": 0.16, + "threshold": -0.7129085659980774, + "operator": "ge", + "actual_fraction": 0.16, + "metric": 0.8966666666666666, + "rescued": 2, + "harmed": 109, + "missed_rescue": 0, + "avoided_harm": 415, + "selected_both_correct": 66, + "selected_both_wrong": 15 + }, + { + "budget": 0.17, + "threshold": -0.7240402102470398, + "operator": "ge", + "actual_fraction": 0.17, + "metric": 0.8891666666666667, + "rescued": 2, + "harmed": 118, + "missed_rescue": 0, + "avoided_harm": 406, + "selected_both_correct": 69, + "selected_both_wrong": 15 + }, + { + "budget": 0.18, + "threshold": -0.7345248460769653, + "operator": "ge", + "actual_fraction": 0.18, + "metric": 0.8833333333333333, + "rescued": 2, + "harmed": 125, + "missed_rescue": 0, + "avoided_harm": 399, + "selected_both_correct": 74, + "selected_both_wrong": 15 + }, + { + "budget": 0.19, + "threshold": -0.7443157434463501, + "operator": "ge", + "actual_fraction": 0.19, + "metric": 0.8775, + "rescued": 2, + "harmed": 132, + "missed_rescue": 0, + "avoided_harm": 392, + "selected_both_correct": 79, + "selected_both_wrong": 15 + }, + { + "budget": 0.2, + "threshold": -0.7533873319625854, + "operator": "ge", + "actual_fraction": 0.2, + "metric": 0.8716666666666667, + "rescued": 2, + "harmed": 139, + "missed_rescue": 0, + "avoided_harm": 385, + "selected_both_correct": 84, + "selected_both_wrong": 15 + }, + { + "budget": 0.21, + "threshold": -0.7606735825538635, + "operator": "ge", + "actual_fraction": 0.21, + "metric": 0.8666666666666667, + "rescued": 2, + "harmed": 145, + "missed_rescue": 0, + "avoided_harm": 379, + "selected_both_correct": 90, + "selected_both_wrong": 15 + }, + { + "budget": 0.22, + "threshold": -0.7657199501991272, + "operator": "ge", + "actual_fraction": 0.22, + "metric": 0.8641666666666666, + "rescued": 2, + "harmed": 148, + "missed_rescue": 0, + "avoided_harm": 376, + "selected_both_correct": 99, + "selected_both_wrong": 15 + }, + { + "budget": 0.23, + "threshold": -0.7712029218673706, + "operator": "ge", + "actual_fraction": 0.23, + "metric": 0.86, + "rescued": 2, + "harmed": 153, + "missed_rescue": 0, + "avoided_harm": 371, + "selected_both_correct": 106, + "selected_both_wrong": 15 + }, + { + "budget": 0.24, + "threshold": -0.7754930257797241, + "operator": "ge", + "actual_fraction": 0.24, + "metric": 0.855, + "rescued": 2, + "harmed": 159, + "missed_rescue": 0, + "avoided_harm": 365, + "selected_both_correct": 112, + "selected_both_wrong": 15 + }, + { + "budget": 0.25, + "threshold": -0.7811619639396667, + "operator": "ge", + "actual_fraction": 0.25, + "metric": 0.8475, + "rescued": 2, + "harmed": 168, + "missed_rescue": 0, + "avoided_harm": 356, + "selected_both_correct": 115, + "selected_both_wrong": 15 + }, + { + "budget": 0.26, + "threshold": -0.7841500043869019, + "operator": "ge", + "actual_fraction": 0.26, + "metric": 0.8408333333333333, + "rescued": 2, + "harmed": 176, + "missed_rescue": 0, + "avoided_harm": 348, + "selected_both_correct": 119, + "selected_both_wrong": 15 + }, + { + "budget": 0.27, + "threshold": -0.78773033618927, + "operator": "ge", + "actual_fraction": 0.27, + "metric": 0.8358333333333333, + "rescued": 2, + "harmed": 182, + "missed_rescue": 0, + "avoided_harm": 342, + "selected_both_correct": 125, + "selected_both_wrong": 15 + }, + { + "budget": 0.28, + "threshold": -0.7923226356506348, + "operator": "ge", + "actual_fraction": 0.28, + "metric": 0.8316666666666667, + "rescued": 2, + "harmed": 187, + "missed_rescue": 0, + "avoided_harm": 337, + "selected_both_correct": 132, + "selected_both_wrong": 15 + }, + { + "budget": 0.29, + "threshold": -0.7950519323348999, + "operator": "ge", + "actual_fraction": 0.29, + "metric": 0.8291666666666667, + "rescued": 2, + "harmed": 190, + "missed_rescue": 0, + "avoided_harm": 334, + "selected_both_correct": 141, + "selected_both_wrong": 15 + }, + { + "budget": 0.3, + "threshold": -0.800890326499939, + "operator": "ge", + "actual_fraction": 0.3, + "metric": 0.825, + "rescued": 2, + "harmed": 195, + "missed_rescue": 0, + "avoided_harm": 329, + "selected_both_correct": 148, + "selected_both_wrong": 15 + }, + { + "budget": 0.31, + "threshold": -0.8055772185325623, + "operator": "ge", + "actual_fraction": 0.31, + "metric": 0.8208333333333333, + "rescued": 2, + "harmed": 200, + "missed_rescue": 0, + "avoided_harm": 324, + "selected_both_correct": 155, + "selected_both_wrong": 15 + }, + { + "budget": 0.32, + "threshold": -0.809782087802887, + "operator": "ge", + "actual_fraction": 0.32, + "metric": 0.8183333333333334, + "rescued": 2, + "harmed": 203, + "missed_rescue": 0, + "avoided_harm": 321, + "selected_both_correct": 164, + "selected_both_wrong": 15 + }, + { + "budget": 0.33, + "threshold": -0.8134240508079529, + "operator": "ge", + "actual_fraction": 0.33, + "metric": 0.8125, + "rescued": 2, + "harmed": 210, + "missed_rescue": 0, + "avoided_harm": 314, + "selected_both_correct": 169, + "selected_both_wrong": 15 + }, + { + "budget": 0.34, + "threshold": -0.8167235255241394, + "operator": "ge", + "actual_fraction": 0.34, + "metric": 0.81, + "rescued": 2, + "harmed": 213, + "missed_rescue": 0, + "avoided_harm": 311, + "selected_both_correct": 178, + "selected_both_wrong": 15 + }, + { + "budget": 0.35, + "threshold": -0.8208380937576294, + "operator": "ge", + "actual_fraction": 0.35, + "metric": 0.8041666666666667, + "rescued": 2, + "harmed": 220, + "missed_rescue": 0, + "avoided_harm": 304, + "selected_both_correct": 183, + "selected_both_wrong": 15 + }, + { + "budget": 0.36, + "threshold": -0.8250406384468079, + "operator": "ge", + "actual_fraction": 0.36, + "metric": 0.8016666666666666, + "rescued": 2, + "harmed": 223, + "missed_rescue": 0, + "avoided_harm": 301, + "selected_both_correct": 192, + "selected_both_wrong": 15 + }, + { + "budget": 0.37, + "threshold": -0.8295461535453796, + "operator": "ge", + "actual_fraction": 0.37, + "metric": 0.7966666666666666, + "rescued": 2, + "harmed": 229, + "missed_rescue": 0, + "avoided_harm": 295, + "selected_both_correct": 198, + "selected_both_wrong": 15 + }, + { + "budget": 0.38, + "threshold": -0.8328037261962891, + "operator": "ge", + "actual_fraction": 0.38, + "metric": 0.7916666666666666, + "rescued": 2, + "harmed": 235, + "missed_rescue": 0, + "avoided_harm": 289, + "selected_both_correct": 204, + "selected_both_wrong": 15 + }, + { + "budget": 0.39, + "threshold": -0.8359338641166687, + "operator": "ge", + "actual_fraction": 0.39, + "metric": 0.7875, + "rescued": 2, + "harmed": 240, + "missed_rescue": 0, + "avoided_harm": 284, + "selected_both_correct": 211, + "selected_both_wrong": 15 + }, + { + "budget": 0.4, + "threshold": -0.8391940593719482, + "operator": "ge", + "actual_fraction": 0.4, + "metric": 0.78, + "rescued": 2, + "harmed": 249, + "missed_rescue": 0, + "avoided_harm": 275, + "selected_both_correct": 214, + "selected_both_wrong": 15 + }, + { + "budget": 0.41, + "threshold": -0.8424465656280518, + "operator": "ge", + "actual_fraction": 0.4091666666666667, + "metric": 0.7775, + "rescued": 2, + "harmed": 252, + "missed_rescue": 0, + "avoided_harm": 272, + "selected_both_correct": 222, + "selected_both_wrong": 15 + }, + { + "budget": 0.42, + "threshold": -0.8460794687271118, + "operator": "ge", + "actual_fraction": 0.42, + "metric": 0.7741666666666667, + "rescued": 2, + "harmed": 256, + "missed_rescue": 0, + "avoided_harm": 268, + "selected_both_correct": 231, + "selected_both_wrong": 15 + }, + { + "budget": 0.43, + "threshold": -0.8500295281410217, + "operator": "ge", + "actual_fraction": 0.43, + "metric": 0.7716666666666666, + "rescued": 2, + "harmed": 259, + "missed_rescue": 0, + "avoided_harm": 265, + "selected_both_correct": 240, + "selected_both_wrong": 15 + }, + { + "budget": 0.44, + "threshold": -0.852453887462616, + "operator": "ge", + "actual_fraction": 0.44, + "metric": 0.7666666666666667, + "rescued": 2, + "harmed": 265, + "missed_rescue": 0, + "avoided_harm": 259, + "selected_both_correct": 246, + "selected_both_wrong": 15 + }, + { + "budget": 0.45, + "threshold": -0.8539360165596008, + "operator": "ge", + "actual_fraction": 0.45, + "metric": 0.7625, + "rescued": 2, + "harmed": 270, + "missed_rescue": 0, + "avoided_harm": 254, + "selected_both_correct": 253, + "selected_both_wrong": 15 + }, + { + "budget": 0.46, + "threshold": -0.8562509417533875, + "operator": "ge", + "actual_fraction": 0.46, + "metric": 0.76, + "rescued": 2, + "harmed": 273, + "missed_rescue": 0, + "avoided_harm": 251, + "selected_both_correct": 262, + "selected_both_wrong": 15 + }, + { + "budget": 0.47, + "threshold": -0.8592684864997864, + "operator": "ge", + "actual_fraction": 0.47, + "metric": 0.7558333333333334, + "rescued": 2, + "harmed": 278, + "missed_rescue": 0, + "avoided_harm": 246, + "selected_both_correct": 269, + "selected_both_wrong": 15 + }, + { + "budget": 0.48, + "threshold": -0.8621411919593811, + "operator": "ge", + "actual_fraction": 0.48, + "metric": 0.7525, + "rescued": 2, + "harmed": 282, + "missed_rescue": 0, + "avoided_harm": 242, + "selected_both_correct": 277, + "selected_both_wrong": 15 + }, + { + "budget": 0.49, + "threshold": -0.863978385925293, + "operator": "ge", + "actual_fraction": 0.49, + "metric": 0.7483333333333333, + "rescued": 2, + "harmed": 287, + "missed_rescue": 0, + "avoided_harm": 237, + "selected_both_correct": 284, + "selected_both_wrong": 15 + }, + { + "budget": 0.5, + "threshold": -0.8661247491836548, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.7425, + "rescued": 2, + "harmed": 294, + "missed_rescue": 0, + "avoided_harm": 230, + "selected_both_correct": 289, + "selected_both_wrong": 15 + }, + { + "budget": 0.51, + "threshold": -0.8688956499099731, + "operator": "ge", + "actual_fraction": 0.51, + "metric": 0.7383333333333333, + "rescued": 2, + "harmed": 299, + "missed_rescue": 0, + "avoided_harm": 225, + "selected_both_correct": 296, + "selected_both_wrong": 15 + }, + { + "budget": 0.52, + "threshold": -0.8706188201904297, + "operator": "ge", + "actual_fraction": 0.52, + "metric": 0.7358333333333333, + "rescued": 2, + "harmed": 302, + "missed_rescue": 0, + "avoided_harm": 222, + "selected_both_correct": 305, + "selected_both_wrong": 15 + }, + { + "budget": 0.53, + "threshold": -0.8727357983589172, + "operator": "ge", + "actual_fraction": 0.53, + "metric": 0.73, + "rescued": 2, + "harmed": 309, + "missed_rescue": 0, + "avoided_harm": 215, + "selected_both_correct": 310, + "selected_both_wrong": 15 + }, + { + "budget": 0.54, + "threshold": -0.874894380569458, + "operator": "ge", + "actual_fraction": 0.54, + "metric": 0.725, + "rescued": 2, + "harmed": 315, + "missed_rescue": 0, + "avoided_harm": 209, + "selected_both_correct": 316, + "selected_both_wrong": 15 + }, + { + "budget": 0.55, + "threshold": -0.8764559626579285, + "operator": "ge", + "actual_fraction": 0.55, + "metric": 0.72, + "rescued": 2, + "harmed": 321, + "missed_rescue": 0, + "avoided_harm": 203, + "selected_both_correct": 322, + "selected_both_wrong": 15 + }, + { + "budget": 0.56, + "threshold": -0.879269540309906, + "operator": "ge", + "actual_fraction": 0.56, + "metric": 0.7166666666666667, + "rescued": 2, + "harmed": 325, + "missed_rescue": 0, + "avoided_harm": 199, + "selected_both_correct": 330, + "selected_both_wrong": 15 + }, + { + "budget": 0.57, + "threshold": -0.881222128868103, + "operator": "ge", + "actual_fraction": 0.5691666666666667, + "metric": 0.7116666666666667, + "rescued": 2, + "harmed": 331, + "missed_rescue": 0, + "avoided_harm": 193, + "selected_both_correct": 335, + "selected_both_wrong": 15 + }, + { + "budget": 0.58, + "threshold": -0.882451593875885, + "operator": "ge", + "actual_fraction": 0.58, + "metric": 0.7075, + "rescued": 2, + "harmed": 336, + "missed_rescue": 0, + "avoided_harm": 188, + "selected_both_correct": 343, + "selected_both_wrong": 15 + }, + { + "budget": 0.59, + "threshold": -0.8843295574188232, + "operator": "ge", + "actual_fraction": 0.59, + "metric": 0.7025, + "rescued": 2, + "harmed": 342, + "missed_rescue": 0, + "avoided_harm": 182, + "selected_both_correct": 349, + "selected_both_wrong": 15 + }, + { + "budget": 0.6, + "threshold": -0.8863861560821533, + "operator": "ge", + "actual_fraction": 0.6, + "metric": 0.6983333333333334, + "rescued": 2, + "harmed": 347, + "missed_rescue": 0, + "avoided_harm": 177, + "selected_both_correct": 356, + "selected_both_wrong": 15 + }, + { + "budget": 0.61, + "threshold": -0.8876380920410156, + "operator": "ge", + "actual_fraction": 0.61, + "metric": 0.6933333333333334, + "rescued": 2, + "harmed": 353, + "missed_rescue": 0, + "avoided_harm": 171, + "selected_both_correct": 362, + "selected_both_wrong": 15 + }, + { + "budget": 0.62, + "threshold": -0.8888679146766663, + "operator": "ge", + "actual_fraction": 0.62, + "metric": 0.69, + "rescued": 2, + "harmed": 357, + "missed_rescue": 0, + "avoided_harm": 167, + "selected_both_correct": 370, + "selected_both_wrong": 15 + }, + { + "budget": 0.63, + "threshold": -0.8905391693115234, + "operator": "ge", + "actual_fraction": 0.63, + "metric": 0.6883333333333334, + "rescued": 2, + "harmed": 359, + "missed_rescue": 0, + "avoided_harm": 165, + "selected_both_correct": 380, + "selected_both_wrong": 15 + }, + { + "budget": 0.64, + "threshold": -0.8919612169265747, + "operator": "ge", + "actual_fraction": 0.64, + "metric": 0.6875, + "rescued": 2, + "harmed": 360, + "missed_rescue": 0, + "avoided_harm": 164, + "selected_both_correct": 391, + "selected_both_wrong": 15 + }, + { + "budget": 0.65, + "threshold": -0.8940081596374512, + "operator": "ge", + "actual_fraction": 0.65, + "metric": 0.6841666666666667, + "rescued": 2, + "harmed": 364, + "missed_rescue": 0, + "avoided_harm": 160, + "selected_both_correct": 399, + "selected_both_wrong": 15 + }, + { + "budget": 0.66, + "threshold": -0.895609974861145, + "operator": "ge", + "actual_fraction": 0.66, + "metric": 0.68, + "rescued": 2, + "harmed": 369, + "missed_rescue": 0, + "avoided_harm": 155, + "selected_both_correct": 406, + "selected_both_wrong": 15 + }, + { + "budget": 0.67, + "threshold": -0.8972456455230713, + "operator": "ge", + "actual_fraction": 0.67, + "metric": 0.6733333333333333, + "rescued": 2, + "harmed": 377, + "missed_rescue": 0, + "avoided_harm": 147, + "selected_both_correct": 410, + "selected_both_wrong": 15 + }, + { + "budget": 0.68, + "threshold": -0.8986453413963318, + "operator": "ge", + "actual_fraction": 0.68, + "metric": 0.6716666666666666, + "rescued": 2, + "harmed": 379, + "missed_rescue": 0, + "avoided_harm": 145, + "selected_both_correct": 420, + "selected_both_wrong": 15 + }, + { + "budget": 0.69, + "threshold": -0.8996874094009399, + "operator": "ge", + "actual_fraction": 0.6891666666666667, + "metric": 0.67, + "rescued": 2, + "harmed": 381, + "missed_rescue": 0, + "avoided_harm": 143, + "selected_both_correct": 429, + "selected_both_wrong": 15 + }, + { + "budget": 0.7, + "threshold": -0.9008907079696655, + "operator": "ge", + "actual_fraction": 0.7, + "metric": 0.665, + "rescued": 2, + "harmed": 387, + "missed_rescue": 0, + "avoided_harm": 137, + "selected_both_correct": 436, + "selected_both_wrong": 15 + }, + { + "budget": 0.71, + "threshold": -0.9031620621681213, + "operator": "ge", + "actual_fraction": 0.71, + "metric": 0.6583333333333333, + "rescued": 2, + "harmed": 395, + "missed_rescue": 0, + "avoided_harm": 129, + "selected_both_correct": 440, + "selected_both_wrong": 15 + }, + { + "budget": 0.72, + "threshold": -0.9056642055511475, + "operator": "ge", + "actual_fraction": 0.72, + "metric": 0.6541666666666667, + "rescued": 2, + "harmed": 400, + "missed_rescue": 0, + "avoided_harm": 124, + "selected_both_correct": 447, + "selected_both_wrong": 15 + }, + { + "budget": 0.73, + "threshold": -0.9068957567214966, + "operator": "ge", + "actual_fraction": 0.73, + "metric": 0.6508333333333334, + "rescued": 2, + "harmed": 404, + "missed_rescue": 0, + "avoided_harm": 120, + "selected_both_correct": 455, + "selected_both_wrong": 15 + }, + { + "budget": 0.74, + "threshold": -0.9093236327171326, + "operator": "ge", + "actual_fraction": 0.74, + "metric": 0.6491666666666667, + "rescued": 2, + "harmed": 406, + "missed_rescue": 0, + "avoided_harm": 118, + "selected_both_correct": 465, + "selected_both_wrong": 15 + }, + { + "budget": 0.75, + "threshold": -0.9112368822097778, + "operator": "ge", + "actual_fraction": 0.75, + "metric": 0.645, + "rescued": 2, + "harmed": 411, + "missed_rescue": 0, + "avoided_harm": 113, + "selected_both_correct": 472, + "selected_both_wrong": 15 + }, + { + "budget": 0.76, + "threshold": -0.9129221439361572, + "operator": "ge", + "actual_fraction": 0.76, + "metric": 0.6425, + "rescued": 2, + "harmed": 414, + "missed_rescue": 0, + "avoided_harm": 110, + "selected_both_correct": 481, + "selected_both_wrong": 15 + }, + { + "budget": 0.77, + "threshold": -0.9144289493560791, + "operator": "ge", + "actual_fraction": 0.77, + "metric": 0.64, + "rescued": 2, + "harmed": 417, + "missed_rescue": 0, + "avoided_harm": 107, + "selected_both_correct": 490, + "selected_both_wrong": 15 + }, + { + "budget": 0.78, + "threshold": -0.9158422946929932, + "operator": "ge", + "actual_fraction": 0.78, + "metric": 0.6383333333333333, + "rescued": 2, + "harmed": 419, + "missed_rescue": 0, + "avoided_harm": 105, + "selected_both_correct": 500, + "selected_both_wrong": 15 + }, + { + "budget": 0.79, + "threshold": -0.9167781472206116, + "operator": "ge", + "actual_fraction": 0.79, + "metric": 0.6341666666666667, + "rescued": 2, + "harmed": 424, + "missed_rescue": 0, + "avoided_harm": 100, + "selected_both_correct": 507, + "selected_both_wrong": 15 + }, + { + "budget": 0.8, + "threshold": -0.9187189936637878, + "operator": "ge", + "actual_fraction": 0.8, + "metric": 0.6283333333333333, + "rescued": 2, + "harmed": 431, + "missed_rescue": 0, + "avoided_harm": 93, + "selected_both_correct": 512, + "selected_both_wrong": 15 + }, + { + "budget": 0.81, + "threshold": -0.9207327365875244, + "operator": "ge", + "actual_fraction": 0.81, + "metric": 0.625, + "rescued": 2, + "harmed": 435, + "missed_rescue": 0, + "avoided_harm": 89, + "selected_both_correct": 520, + "selected_both_wrong": 15 + }, + { + "budget": 0.82, + "threshold": -0.9226345419883728, + "operator": "ge", + "actual_fraction": 0.8191666666666667, + "metric": 0.6216666666666667, + "rescued": 2, + "harmed": 439, + "missed_rescue": 0, + "avoided_harm": 85, + "selected_both_correct": 527, + "selected_both_wrong": 15 + }, + { + "budget": 0.83, + "threshold": -0.9243993759155273, + "operator": "ge", + "actual_fraction": 0.83, + "metric": 0.6191666666666666, + "rescued": 2, + "harmed": 442, + "missed_rescue": 0, + "avoided_harm": 82, + "selected_both_correct": 537, + "selected_both_wrong": 15 + }, + { + "budget": 0.84, + "threshold": -0.9264947175979614, + "operator": "ge", + "actual_fraction": 0.84, + "metric": 0.6166666666666667, + "rescued": 2, + "harmed": 445, + "missed_rescue": 0, + "avoided_harm": 79, + "selected_both_correct": 546, + "selected_both_wrong": 15 + }, + { + "budget": 0.85, + "threshold": -0.9274600148200989, + "operator": "ge", + "actual_fraction": 0.85, + "metric": 0.6141666666666666, + "rescued": 2, + "harmed": 448, + "missed_rescue": 0, + "avoided_harm": 76, + "selected_both_correct": 555, + "selected_both_wrong": 15 + }, + { + "budget": 0.86, + "threshold": -0.930335283279419, + "operator": "ge", + "actual_fraction": 0.86, + "metric": 0.6108333333333333, + "rescued": 2, + "harmed": 452, + "missed_rescue": 0, + "avoided_harm": 72, + "selected_both_correct": 563, + "selected_both_wrong": 15 + }, + { + "budget": 0.87, + "threshold": -0.9319414496421814, + "operator": "ge", + "actual_fraction": 0.87, + "metric": 0.6066666666666667, + "rescued": 2, + "harmed": 457, + "missed_rescue": 0, + "avoided_harm": 67, + "selected_both_correct": 570, + "selected_both_wrong": 15 + }, + { + "budget": 0.88, + "threshold": -0.9340114593505859, + "operator": "ge", + "actual_fraction": 0.88, + "metric": 0.6033333333333334, + "rescued": 2, + "harmed": 461, + "missed_rescue": 0, + "avoided_harm": 63, + "selected_both_correct": 578, + "selected_both_wrong": 15 + }, + { + "budget": 0.89, + "threshold": -0.9363258481025696, + "operator": "ge", + "actual_fraction": 0.89, + "metric": 0.5983333333333334, + "rescued": 2, + "harmed": 467, + "missed_rescue": 0, + "avoided_harm": 57, + "selected_both_correct": 584, + "selected_both_wrong": 15 + }, + { + "budget": 0.9, + "threshold": -0.9400071501731873, + "operator": "ge", + "actual_fraction": 0.9, + "metric": 0.595, + "rescued": 2, + "harmed": 471, + "missed_rescue": 0, + "avoided_harm": 53, + "selected_both_correct": 592, + "selected_both_wrong": 15 + }, + { + "budget": 0.91, + "threshold": -0.9423198103904724, + "operator": "ge", + "actual_fraction": 0.91, + "metric": 0.5908333333333333, + "rescued": 2, + "harmed": 476, + "missed_rescue": 0, + "avoided_harm": 48, + "selected_both_correct": 599, + "selected_both_wrong": 15 + }, + { + "budget": 0.92, + "threshold": -0.9443356394767761, + "operator": "ge", + "actual_fraction": 0.92, + "metric": 0.5866666666666667, + "rescued": 2, + "harmed": 481, + "missed_rescue": 0, + "avoided_harm": 43, + "selected_both_correct": 606, + "selected_both_wrong": 15 + }, + { + "budget": 0.93, + "threshold": -0.9468213319778442, + "operator": "ge", + "actual_fraction": 0.93, + "metric": 0.5833333333333334, + "rescued": 2, + "harmed": 485, + "missed_rescue": 0, + "avoided_harm": 39, + "selected_both_correct": 614, + "selected_both_wrong": 15 + }, + { + "budget": 0.94, + "threshold": -0.9496435523033142, + "operator": "ge", + "actual_fraction": 0.94, + "metric": 0.5791666666666667, + "rescued": 2, + "harmed": 490, + "missed_rescue": 0, + "avoided_harm": 34, + "selected_both_correct": 621, + "selected_both_wrong": 15 + }, + { + "budget": 0.95, + "threshold": -0.9529175758361816, + "operator": "ge", + "actual_fraction": 0.95, + "metric": 0.5766666666666667, + "rescued": 2, + "harmed": 493, + "missed_rescue": 0, + "avoided_harm": 31, + "selected_both_correct": 630, + "selected_both_wrong": 15 + }, + { + "budget": 0.96, + "threshold": -0.9560379981994629, + "operator": "ge", + "actual_fraction": 0.96, + "metric": 0.57, + "rescued": 2, + "harmed": 501, + "missed_rescue": 0, + "avoided_harm": 23, + "selected_both_correct": 634, + "selected_both_wrong": 15 + }, + { + "budget": 0.97, + "threshold": -0.9591385126113892, + "operator": "ge", + "actual_fraction": 0.97, + "metric": 0.5666666666666667, + "rescued": 2, + "harmed": 505, + "missed_rescue": 0, + "avoided_harm": 19, + "selected_both_correct": 642, + "selected_both_wrong": 15 + }, + { + "budget": 0.98, + "threshold": -0.9627243280410767, + "operator": "ge", + "actual_fraction": 0.98, + "metric": 0.56, + "rescued": 2, + "harmed": 513, + "missed_rescue": 0, + "avoided_harm": 11, + "selected_both_correct": 646, + "selected_both_wrong": 15 + }, + { + "budget": 0.99, + "threshold": -0.9702208638191223, + "operator": "ge", + "actual_fraction": 0.99, + "metric": 0.5558333333333333, + "rescued": 2, + "harmed": 518, + "missed_rescue": 0, + "avoided_harm": 6, + "selected_both_correct": 653, + "selected_both_wrong": 15 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5508333333333333, + "rescued": 2, + "harmed": 524, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 659, + "selected_both_wrong": 15 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.81429064549534, + "rescued": 0, + "harmed": 0, + "missed_rescue": 365, + "avoided_harm": 1907, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.113565757680359, + "metric": 0.8158439765274422, + "rescued": 132, + "harmed": 123, + "missed_rescue": 233, + "avoided_harm": 1784, + "selected_both_correct": 108, + "selected_both_wrong": 295 + }, + { + "budget": 0.02, + "threshold": -0.3627302646636963, + "operator": "ge", + "actual_fraction": 0.15947531929582326, + "metric": 0.8099758370728339, + "rescued": 182, + "harmed": 207, + "missed_rescue": 183, + "avoided_harm": 1700, + "selected_both_correct": 170, + "selected_both_wrong": 365 + }, + { + "budget": 0.03, + "threshold": -0.4222676455974579, + "operator": "ge", + "actual_fraction": 0.2143596824301001, + "metric": 0.798584742837418, + "rescued": 226, + "harmed": 317, + "missed_rescue": 139, + "avoided_harm": 1590, + "selected_both_correct": 261, + "selected_both_wrong": 438 + }, + { + "budget": 0.04, + "threshold": -0.46618229150772095, + "operator": "ge", + "actual_fraction": 0.2545736969278564, + "metric": 0.789264756644805, + "rescued": 257, + "harmed": 402, + "missed_rescue": 108, + "avoided_harm": 1505, + "selected_both_correct": 329, + "selected_both_wrong": 487 + }, + { + "budget": 0.05, + "threshold": -0.5066347122192383, + "operator": "ge", + "actual_fraction": 0.29375215740421123, + "metric": 0.7773558853986883, + "rescued": 277, + "harmed": 491, + "missed_rescue": 88, + "avoided_harm": 1416, + "selected_both_correct": 417, + "selected_both_wrong": 517 + }, + { + "budget": 0.06, + "threshold": -0.5393391251564026, + "operator": "ge", + "actual_fraction": 0.3208491542975492, + "metric": 0.7709699689333793, + "rescued": 291, + "harmed": 542, + "missed_rescue": 74, + "avoided_harm": 1365, + "selected_both_correct": 480, + "selected_both_wrong": 546 + }, + { + "budget": 0.07, + "threshold": -0.5679479837417603, + "operator": "ge", + "actual_fraction": 0.350707628581291, + "metric": 0.7604418363824647, + "rescued": 303, + "harmed": 615, + "missed_rescue": 62, + "avoided_harm": 1292, + "selected_both_correct": 537, + "selected_both_wrong": 577 + }, + { + "budget": 0.08, + "threshold": -0.5907968282699585, + "operator": "ge", + "actual_fraction": 0.36882982395581637, + "metric": 0.7540559199171557, + "rescued": 312, + "harmed": 661, + "missed_rescue": 53, + "avoided_harm": 1246, + "selected_both_correct": 576, + "selected_both_wrong": 588 + }, + { + "budget": 0.09, + "threshold": -0.6168767809867859, + "operator": "ge", + "actual_fraction": 0.391439420089748, + "metric": 0.7452537107352434, + "rescued": 317, + "harmed": 717, + "missed_rescue": 48, + "avoided_harm": 1190, + "selected_both_correct": 627, + "selected_both_wrong": 607 + }, + { + "budget": 0.1, + "threshold": -0.6361178159713745, + "operator": "ge", + "actual_fraction": 0.4138764238867794, + "metric": 0.7392129789437349, + "rescued": 327, + "harmed": 762, + "missed_rescue": 38, + "avoided_harm": 1145, + "selected_both_correct": 688, + "selected_both_wrong": 621 + }, + { + "budget": 0.11, + "threshold": -0.6547113656997681, + "operator": "ge", + "actual_fraction": 0.43372454263030724, + "metric": 0.732827062478426, + "rescued": 336, + "harmed": 808, + "missed_rescue": 29, + "avoided_harm": 1099, + "selected_both_correct": 739, + "selected_both_wrong": 630 + }, + { + "budget": 0.12, + "threshold": -0.6680066585540771, + "operator": "ge", + "actual_fraction": 0.4480497065930273, + "metric": 0.7267863306869174, + "rescued": 338, + "harmed": 845, + "missed_rescue": 27, + "avoided_harm": 1062, + "selected_both_correct": 780, + "selected_both_wrong": 633 + }, + { + "budget": 0.13, + "threshold": -0.6809748411178589, + "operator": "ge", + "actual_fraction": 0.4630652399033483, + "metric": 0.7216085605799103, + "rescued": 343, + "harmed": 880, + "missed_rescue": 22, + "avoided_harm": 1027, + "selected_both_correct": 821, + "selected_both_wrong": 639 + }, + { + "budget": 0.14, + "threshold": -0.6963143348693848, + "operator": "ge", + "actual_fraction": 0.479288919571971, + "metric": 0.7152226441146013, + "rescued": 346, + "harmed": 920, + "missed_rescue": 19, + "avoided_harm": 987, + "selected_both_correct": 868, + "selected_both_wrong": 643 + }, + { + "budget": 0.15, + "threshold": -0.7065573930740356, + "operator": "ge", + "actual_fraction": 0.4932688988608906, + "metric": 0.710735243355195, + "rescued": 348, + "harmed": 948, + "missed_rescue": 17, + "avoided_harm": 959, + "selected_both_correct": 911, + "selected_both_wrong": 651 + }, + { + "budget": 0.16, + "threshold": -0.7129085659980774, + "operator": "ge", + "actual_fraction": 0.5008629616845012, + "metric": 0.7086641353123921, + "rescued": 350, + "harmed": 962, + "missed_rescue": 15, + "avoided_harm": 945, + "selected_both_correct": 937, + "selected_both_wrong": 653 + }, + { + "budget": 0.17, + "threshold": -0.7240402102470398, + "operator": "ge", + "actual_fraction": 0.5155333103210218, + "metric": 0.701415257162582, + "rescued": 350, + "harmed": 1004, + "missed_rescue": 15, + "avoided_harm": 903, + "selected_both_correct": 978, + "selected_both_wrong": 655 + }, + { + "budget": 0.18, + "threshold": -0.7345248460769653, + "operator": "ge", + "actual_fraction": 0.5298584742837418, + "metric": 0.6967552640662754, + "rescued": 355, + "harmed": 1036, + "missed_rescue": 10, + "avoided_harm": 871, + "selected_both_correct": 1020, + "selected_both_wrong": 659 + }, + { + "budget": 0.19, + "threshold": -0.7443157434463501, + "operator": "ge", + "actual_fraction": 0.5455643769416638, + "metric": 0.6908871246116672, + "rescued": 355, + "harmed": 1070, + "missed_rescue": 10, + "avoided_harm": 837, + "selected_both_correct": 1072, + "selected_both_wrong": 664 + }, + { + "budget": 0.2, + "threshold": -0.7533873319625854, + "operator": "ge", + "actual_fraction": 0.5560925094925785, + "metric": 0.6872626855367622, + "rescued": 356, + "harmed": 1092, + "missed_rescue": 9, + "avoided_harm": 815, + "selected_both_correct": 1106, + "selected_both_wrong": 668 + }, + { + "budget": 0.21, + "threshold": -0.7606735825538635, + "operator": "ge", + "actual_fraction": 0.5676561960648947, + "metric": 0.6841560234725578, + "rescued": 356, + "harmed": 1110, + "missed_rescue": 9, + "avoided_harm": 797, + "selected_both_correct": 1153, + "selected_both_wrong": 670 + }, + { + "budget": 0.22, + "threshold": -0.7657199501991272, + "operator": "ge", + "actual_fraction": 0.5766309975837073, + "metric": 0.6808767690714532, + "rescued": 356, + "harmed": 1129, + "missed_rescue": 9, + "avoided_harm": 778, + "selected_both_correct": 1184, + "selected_both_wrong": 672 + }, + { + "budget": 0.23, + "threshold": -0.7712029218673706, + "operator": "ge", + "actual_fraction": 0.5861235761132205, + "metric": 0.6779426993441491, + "rescued": 357, + "harmed": 1147, + "missed_rescue": 8, + "avoided_harm": 760, + "selected_both_correct": 1219, + "selected_both_wrong": 673 + }, + { + "budget": 0.24, + "threshold": -0.7754930257797241, + "operator": "ge", + "actual_fraction": 0.5923369002416292, + "metric": 0.6755264066275457, + "rescued": 357, + "harmed": 1161, + "missed_rescue": 8, + "avoided_harm": 746, + "selected_both_correct": 1238, + "selected_both_wrong": 676 + }, + { + "budget": 0.25, + "threshold": -0.7811619639396667, + "operator": "ge", + "actual_fraction": 0.6013117017604418, + "metric": 0.6727649292371418, + "rescued": 358, + "harmed": 1178, + "missed_rescue": 7, + "avoided_harm": 729, + "selected_both_correct": 1272, + "selected_both_wrong": 676 + }, + { + "budget": 0.26, + "threshold": -0.7841500043869019, + "operator": "ge", + "actual_fraction": 0.6052813255091474, + "metric": 0.6712115982050397, + "rescued": 359, + "harmed": 1188, + "missed_rescue": 6, + "avoided_harm": 719, + "selected_both_correct": 1284, + "selected_both_wrong": 676 + }, + { + "budget": 0.27, + "threshold": -0.78773033618927, + "operator": "ge", + "actual_fraction": 0.6123576113220573, + "metric": 0.6681049361408353, + "rescued": 359, + "harmed": 1206, + "missed_rescue": 6, + "avoided_harm": 701, + "selected_both_correct": 1305, + "selected_both_wrong": 678 + }, + { + "budget": 0.28, + "threshold": -0.7923226356506348, + "operator": "ge", + "actual_fraction": 0.6206420434932689, + "metric": 0.664998274076631, + "rescued": 359, + "harmed": 1224, + "missed_rescue": 6, + "avoided_harm": 683, + "selected_both_correct": 1333, + "selected_both_wrong": 680 + }, + { + "budget": 0.29, + "threshold": -0.7950519323348999, + "operator": "ge", + "actual_fraction": 0.6251294442526751, + "metric": 0.6632723507076286, + "rescued": 359, + "harmed": 1234, + "missed_rescue": 6, + "avoided_harm": 673, + "selected_both_correct": 1348, + "selected_both_wrong": 681 + }, + { + "budget": 0.3, + "threshold": -0.800890326499939, + "operator": "ge", + "actual_fraction": 0.6373835001725924, + "metric": 0.6580945806006213, + "rescued": 359, + "harmed": 1264, + "missed_rescue": 6, + "avoided_harm": 643, + "selected_both_correct": 1388, + "selected_both_wrong": 682 + }, + { + "budget": 0.31, + "threshold": -0.8055772185325623, + "operator": "ge", + "actual_fraction": 0.6470486710390059, + "metric": 0.6546427338626165, + "rescued": 359, + "harmed": 1284, + "missed_rescue": 6, + "avoided_harm": 623, + "selected_both_correct": 1423, + "selected_both_wrong": 683 + }, + { + "budget": 0.32, + "threshold": -0.809782087802887, + "operator": "ge", + "actual_fraction": 0.6556782878840179, + "metric": 0.6527442181567138, + "rescued": 360, + "harmed": 1296, + "missed_rescue": 5, + "avoided_harm": 611, + "selected_both_correct": 1457, + "selected_both_wrong": 686 + }, + { + "budget": 0.33, + "threshold": -0.8134240508079529, + "operator": "ge", + "actual_fraction": 0.6636175353814291, + "metric": 0.6498101484294098, + "rescued": 360, + "harmed": 1313, + "missed_rescue": 5, + "avoided_harm": 594, + "selected_both_correct": 1485, + "selected_both_wrong": 687 + }, + { + "budget": 0.34, + "threshold": -0.8167235255241394, + "operator": "ge", + "actual_fraction": 0.6708664135312392, + "metric": 0.6472212633759061, + "rescued": 360, + "harmed": 1328, + "missed_rescue": 5, + "avoided_harm": 579, + "selected_both_correct": 1511, + "selected_both_wrong": 688 + }, + { + "budget": 0.35, + "threshold": -0.8208380937576294, + "operator": "ge", + "actual_fraction": 0.6812219537452537, + "metric": 0.6435968243010011, + "rescued": 360, + "harmed": 1349, + "missed_rescue": 5, + "avoided_harm": 558, + "selected_both_correct": 1549, + "selected_both_wrong": 689 + }, + { + "budget": 0.36, + "threshold": -0.8250406384468079, + "operator": "ge", + "actual_fraction": 0.689333793579565, + "metric": 0.6408353469105972, + "rescued": 360, + "harmed": 1365, + "missed_rescue": 5, + "avoided_harm": 542, + "selected_both_correct": 1578, + "selected_both_wrong": 691 + }, + { + "budget": 0.37, + "threshold": -0.8295461535453796, + "operator": "ge", + "actual_fraction": 0.7017604418363824, + "metric": 0.6347946151190887, + "rescued": 360, + "harmed": 1400, + "missed_rescue": 5, + "avoided_harm": 507, + "selected_both_correct": 1613, + "selected_both_wrong": 693 + }, + { + "budget": 0.38, + "threshold": -0.8328037261962891, + "operator": "ge", + "actual_fraction": 0.7088367276492924, + "metric": 0.6320331377286849, + "rescued": 360, + "harmed": 1416, + "missed_rescue": 5, + "avoided_harm": 491, + "selected_both_correct": 1637, + "selected_both_wrong": 694 + }, + { + "budget": 0.39, + "threshold": -0.8359338641166687, + "operator": "ge", + "actual_fraction": 0.7166033828098033, + "metric": 0.6290990680013807, + "rescued": 360, + "harmed": 1433, + "missed_rescue": 5, + "avoided_harm": 474, + "selected_both_correct": 1665, + "selected_both_wrong": 694 + }, + { + "budget": 0.4, + "threshold": -0.8391940593719482, + "operator": "ge", + "actual_fraction": 0.7238522609596134, + "metric": 0.6265101829478771, + "rescued": 361, + "harmed": 1449, + "missed_rescue": 4, + "avoided_harm": 458, + "selected_both_correct": 1689, + "selected_both_wrong": 695 + }, + { + "budget": 0.41, + "threshold": -0.8424465656280518, + "operator": "ge", + "actual_fraction": 0.7302381774249224, + "metric": 0.6244390749050742, + "rescued": 361, + "harmed": 1461, + "missed_rescue": 4, + "avoided_harm": 446, + "selected_both_correct": 1713, + "selected_both_wrong": 696 + }, + { + "budget": 0.42, + "threshold": -0.8460794687271118, + "operator": "ge", + "actual_fraction": 0.7392129789437349, + "metric": 0.6211598205039697, + "rescued": 361, + "harmed": 1480, + "missed_rescue": 4, + "avoided_harm": 427, + "selected_both_correct": 1746, + "selected_both_wrong": 696 + }, + { + "budget": 0.43, + "threshold": -0.8500295281410217, + "operator": "ge", + "actual_fraction": 0.747152226441146, + "metric": 0.618570935450466, + "rescued": 361, + "harmed": 1495, + "missed_rescue": 4, + "avoided_harm": 412, + "selected_both_correct": 1777, + "selected_both_wrong": 696 + }, + { + "budget": 0.44, + "threshold": -0.852453887462616, + "operator": "ge", + "actual_fraction": 0.7523299965481532, + "metric": 0.6175353814290645, + "rescued": 361, + "harmed": 1501, + "missed_rescue": 4, + "avoided_harm": 406, + "selected_both_correct": 1800, + "selected_both_wrong": 697 + }, + { + "budget": 0.45, + "threshold": -0.8539360165596008, + "operator": "ge", + "actual_fraction": 0.7562996202968588, + "metric": 0.6171901967552641, + "rescued": 361, + "harmed": 1503, + "missed_rescue": 4, + "avoided_harm": 404, + "selected_both_correct": 1821, + "selected_both_wrong": 697 + }, + { + "budget": 0.46, + "threshold": -0.8562509417533875, + "operator": "ge", + "actual_fraction": 0.7613047980669658, + "metric": 0.6159820503969624, + "rescued": 361, + "harmed": 1510, + "missed_rescue": 4, + "avoided_harm": 397, + "selected_both_correct": 1843, + "selected_both_wrong": 697 + }, + { + "budget": 0.47, + "threshold": -0.8592684864997864, + "operator": "ge", + "actual_fraction": 0.7668277528477736, + "metric": 0.6144287193648602, + "rescued": 361, + "harmed": 1519, + "missed_rescue": 4, + "avoided_harm": 388, + "selected_both_correct": 1865, + "selected_both_wrong": 698 + }, + { + "budget": 0.48, + "threshold": -0.8621411919593811, + "operator": "ge", + "actual_fraction": 0.774249223334484, + "metric": 0.6127027959958578, + "rescued": 362, + "harmed": 1530, + "missed_rescue": 3, + "avoided_harm": 377, + "selected_both_correct": 1895, + "selected_both_wrong": 699 + }, + { + "budget": 0.49, + "threshold": -0.863978385925293, + "operator": "ge", + "actual_fraction": 0.7790818087676907, + "metric": 0.6104590956161546, + "rescued": 362, + "harmed": 1543, + "missed_rescue": 3, + "avoided_harm": 364, + "selected_both_correct": 1909, + "selected_both_wrong": 700 + }, + { + "budget": 0.5, + "threshold": -0.8661247491836548, + "operator": "ge", + "actual_fraction": 0.7849499482222989, + "metric": 0.6080428028995513, + "rescued": 362, + "harmed": 1557, + "missed_rescue": 3, + "avoided_harm": 350, + "selected_both_correct": 1928, + "selected_both_wrong": 701 + }, + { + "budget": 0.51, + "threshold": -0.8688956499099731, + "operator": "ge", + "actual_fraction": 0.7928891957197101, + "metric": 0.6056265101829479, + "rescued": 363, + "harmed": 1572, + "missed_rescue": 2, + "avoided_harm": 335, + "selected_both_correct": 1958, + "selected_both_wrong": 701 + }, + { + "budget": 0.52, + "threshold": -0.8706188201904297, + "operator": "ge", + "actual_fraction": 0.7970314118053159, + "metric": 0.6044183638246462, + "rescued": 363, + "harmed": 1579, + "missed_rescue": 2, + "avoided_harm": 328, + "selected_both_correct": 1975, + "selected_both_wrong": 701 + }, + { + "budget": 0.53, + "threshold": -0.8727357983589172, + "operator": "ge", + "actual_fraction": 0.8016914049016224, + "metric": 0.6037279944770452, + "rescued": 363, + "harmed": 1583, + "missed_rescue": 2, + "avoided_harm": 324, + "selected_both_correct": 1998, + "selected_both_wrong": 701 + }, + { + "budget": 0.54, + "threshold": -0.874894380569458, + "operator": "ge", + "actual_fraction": 0.8070417673455299, + "metric": 0.6018294787711426, + "rescued": 363, + "harmed": 1594, + "missed_rescue": 2, + "avoided_harm": 313, + "selected_both_correct": 2018, + "selected_both_wrong": 701 + }, + { + "budget": 0.55, + "threshold": -0.8764559626579285, + "operator": "ge", + "actual_fraction": 0.8120469451156369, + "metric": 0.6007939247497411, + "rescued": 363, + "harmed": 1600, + "missed_rescue": 2, + "avoided_harm": 307, + "selected_both_correct": 2041, + "selected_both_wrong": 701 + }, + { + "budget": 0.56, + "threshold": -0.879269540309906, + "operator": "ge", + "actual_fraction": 0.8180876769071453, + "metric": 0.5990680013807387, + "rescued": 363, + "harmed": 1610, + "missed_rescue": 2, + "avoided_harm": 297, + "selected_both_correct": 2065, + "selected_both_wrong": 702 + }, + { + "budget": 0.57, + "threshold": -0.881222128868103, + "operator": "ge", + "actual_fraction": 0.823610631687953, + "metric": 0.5976872626855367, + "rescued": 363, + "harmed": 1618, + "missed_rescue": 2, + "avoided_harm": 289, + "selected_both_correct": 2086, + "selected_both_wrong": 705 + }, + { + "budget": 0.58, + "threshold": -0.882451593875885, + "operator": "ge", + "actual_fraction": 0.8272350707628582, + "metric": 0.5963065239903348, + "rescued": 363, + "harmed": 1626, + "missed_rescue": 2, + "avoided_harm": 281, + "selected_both_correct": 2099, + "selected_both_wrong": 705 + }, + { + "budget": 0.59, + "threshold": -0.8843295574188232, + "operator": "ge", + "actual_fraction": 0.8313772868484639, + "metric": 0.5957887469796341, + "rescued": 364, + "harmed": 1630, + "missed_rescue": 1, + "avoided_harm": 277, + "selected_both_correct": 2118, + "selected_both_wrong": 705 + }, + { + "budget": 0.6, + "threshold": -0.8863861560821533, + "operator": "ge", + "actual_fraction": 0.8355195029340697, + "metric": 0.5942354159475319, + "rescued": 364, + "harmed": 1639, + "missed_rescue": 1, + "avoided_harm": 268, + "selected_both_correct": 2133, + "selected_both_wrong": 705 + }, + { + "budget": 0.61, + "threshold": -0.8876380920410156, + "operator": "ge", + "actual_fraction": 0.8391439420089748, + "metric": 0.5931998619261305, + "rescued": 364, + "harmed": 1645, + "missed_rescue": 1, + "avoided_harm": 262, + "selected_both_correct": 2148, + "selected_both_wrong": 705 + }, + { + "budget": 0.62, + "threshold": -0.8888679146766663, + "operator": "ge", + "actual_fraction": 0.8425957887469796, + "metric": 0.5923369002416292, + "rescued": 364, + "harmed": 1650, + "missed_rescue": 1, + "avoided_harm": 257, + "selected_both_correct": 2163, + "selected_both_wrong": 705 + }, + { + "budget": 0.63, + "threshold": -0.8905391693115234, + "operator": "ge", + "actual_fraction": 0.8469105971694857, + "metric": 0.5913013462202278, + "rescued": 364, + "harmed": 1656, + "missed_rescue": 1, + "avoided_harm": 251, + "selected_both_correct": 2182, + "selected_both_wrong": 705 + }, + { + "budget": 0.64, + "threshold": -0.8919612169265747, + "operator": "ge", + "actual_fraction": 0.8503624439074905, + "metric": 0.5900931998619261, + "rescued": 364, + "harmed": 1663, + "missed_rescue": 1, + "avoided_harm": 244, + "selected_both_correct": 2195, + "selected_both_wrong": 705 + }, + { + "budget": 0.65, + "threshold": -0.8940081596374512, + "operator": "ge", + "actual_fraction": 0.8560579910251985, + "metric": 0.5881946841560235, + "rescued": 364, + "harmed": 1674, + "missed_rescue": 1, + "avoided_harm": 233, + "selected_both_correct": 2217, + "selected_both_wrong": 705 + }, + { + "budget": 0.66, + "threshold": -0.895609974861145, + "operator": "ge", + "actual_fraction": 0.8603727994477045, + "metric": 0.586468760787021, + "rescued": 365, + "harmed": 1685, + "missed_rescue": 0, + "avoided_harm": 222, + "selected_both_correct": 2230, + "selected_both_wrong": 705 + }, + { + "budget": 0.67, + "threshold": -0.8972456455230713, + "operator": "ge", + "actual_fraction": 0.8643424231964101, + "metric": 0.5861235761132205, + "rescued": 365, + "harmed": 1687, + "missed_rescue": 0, + "avoided_harm": 220, + "selected_both_correct": 2250, + "selected_both_wrong": 706 + }, + { + "budget": 0.68, + "threshold": -0.8986453413963318, + "operator": "ge", + "actual_fraction": 0.8688298239558163, + "metric": 0.5849154297549188, + "rescued": 365, + "harmed": 1694, + "missed_rescue": 0, + "avoided_harm": 213, + "selected_both_correct": 2269, + "selected_both_wrong": 706 + }, + { + "budget": 0.69, + "threshold": -0.8996874094009399, + "operator": "ge", + "actual_fraction": 0.8712461166724197, + "metric": 0.5840524680704177, + "rescued": 365, + "harmed": 1699, + "missed_rescue": 0, + "avoided_harm": 208, + "selected_both_correct": 2278, + "selected_both_wrong": 706 + }, + { + "budget": 0.7, + "threshold": -0.9008907079696655, + "operator": "ge", + "actual_fraction": 0.8746979634104246, + "metric": 0.5826717293752157, + "rescued": 365, + "harmed": 1707, + "missed_rescue": 0, + "avoided_harm": 200, + "selected_both_correct": 2290, + "selected_both_wrong": 706 + }, + { + "budget": 0.71, + "threshold": -0.9031620621681213, + "operator": "ge", + "actual_fraction": 0.8817742492233345, + "metric": 0.5806006213324129, + "rescued": 365, + "harmed": 1719, + "missed_rescue": 0, + "avoided_harm": 188, + "selected_both_correct": 2319, + "selected_both_wrong": 706 + }, + { + "budget": 0.72, + "threshold": -0.9056642055511475, + "operator": "ge", + "actual_fraction": 0.8893683120469451, + "metric": 0.5781843286158095, + "rescued": 365, + "harmed": 1733, + "missed_rescue": 0, + "avoided_harm": 174, + "selected_both_correct": 2348, + "selected_both_wrong": 707 + }, + { + "budget": 0.73, + "threshold": -0.9068957567214966, + "operator": "ge", + "actual_fraction": 0.8933379357956507, + "metric": 0.5773213669313082, + "rescued": 365, + "harmed": 1738, + "missed_rescue": 0, + "avoided_harm": 169, + "selected_both_correct": 2366, + "selected_both_wrong": 707 + }, + { + "budget": 0.74, + "threshold": -0.9093236327171326, + "operator": "ge", + "actual_fraction": 0.8981705212288574, + "metric": 0.5762858129099068, + "rescued": 365, + "harmed": 1744, + "missed_rescue": 0, + "avoided_harm": 163, + "selected_both_correct": 2388, + "selected_both_wrong": 707 + }, + { + "budget": 0.75, + "threshold": -0.9112368822097778, + "operator": "ge", + "actual_fraction": 0.9023127373144633, + "metric": 0.5754228512254056, + "rescued": 365, + "harmed": 1749, + "missed_rescue": 0, + "avoided_harm": 158, + "selected_both_correct": 2407, + "selected_both_wrong": 707 + }, + { + "budget": 0.76, + "threshold": -0.9129221439361572, + "operator": "ge", + "actual_fraction": 0.9081808767690714, + "metric": 0.5738695201933034, + "rescued": 365, + "harmed": 1758, + "missed_rescue": 0, + "avoided_harm": 149, + "selected_both_correct": 2431, + "selected_both_wrong": 708 + }, + { + "budget": 0.77, + "threshold": -0.9144289493560791, + "operator": "ge", + "actual_fraction": 0.9123230928546773, + "metric": 0.5723161891612012, + "rescued": 365, + "harmed": 1767, + "missed_rescue": 0, + "avoided_harm": 140, + "selected_both_correct": 2446, + "selected_both_wrong": 708 + }, + { + "budget": 0.78, + "threshold": -0.9158422946929932, + "operator": "ge", + "actual_fraction": 0.9149119779081809, + "metric": 0.5711080428028995, + "rescued": 365, + "harmed": 1774, + "missed_rescue": 0, + "avoided_harm": 133, + "selected_both_correct": 2454, + "selected_both_wrong": 708 + }, + { + "budget": 0.79, + "threshold": -0.9167781472206116, + "operator": "ge", + "actual_fraction": 0.9168104936140835, + "metric": 0.5705902657921988, + "rescued": 365, + "harmed": 1777, + "missed_rescue": 0, + "avoided_harm": 130, + "selected_both_correct": 2462, + "selected_both_wrong": 708 + }, + { + "budget": 0.8, + "threshold": -0.9187189936637878, + "operator": "ge", + "actual_fraction": 0.922851225405592, + "metric": 0.5683465654124957, + "rescued": 365, + "harmed": 1790, + "missed_rescue": 0, + "avoided_harm": 117, + "selected_both_correct": 2484, + "selected_both_wrong": 708 + }, + { + "budget": 0.81, + "threshold": -0.9207327365875244, + "operator": "ge", + "actual_fraction": 0.9280289955125992, + "metric": 0.5667932343803935, + "rescued": 365, + "harmed": 1799, + "missed_rescue": 0, + "avoided_harm": 108, + "selected_both_correct": 2505, + "selected_both_wrong": 708 + }, + { + "budget": 0.82, + "threshold": -0.9226345419883728, + "operator": "ge", + "actual_fraction": 0.9342423196410079, + "metric": 0.5655850880220918, + "rescued": 365, + "harmed": 1806, + "missed_rescue": 0, + "avoided_harm": 101, + "selected_both_correct": 2534, + "selected_both_wrong": 708 + }, + { + "budget": 0.83, + "threshold": -0.9243993759155273, + "operator": "ge", + "actual_fraction": 0.9387297204004142, + "metric": 0.5642043493268899, + "rescued": 365, + "harmed": 1814, + "missed_rescue": 0, + "avoided_harm": 93, + "selected_both_correct": 2552, + "selected_both_wrong": 708 + }, + { + "budget": 0.84, + "threshold": -0.9264947175979614, + "operator": "ge", + "actual_fraction": 0.9440800828443218, + "metric": 0.5626510182947877, + "rescued": 365, + "harmed": 1823, + "missed_rescue": 0, + "avoided_harm": 84, + "selected_both_correct": 2574, + "selected_both_wrong": 708 + }, + { + "budget": 0.85, + "threshold": -0.9274600148200989, + "operator": "ge", + "actual_fraction": 0.9463237832240249, + "metric": 0.5624784259578874, + "rescued": 365, + "harmed": 1824, + "missed_rescue": 0, + "avoided_harm": 83, + "selected_both_correct": 2586, + "selected_both_wrong": 708 + }, + { + "budget": 0.86, + "threshold": -0.930335283279419, + "operator": "ge", + "actual_fraction": 0.9534000690369347, + "metric": 0.5607525025888851, + "rescued": 365, + "harmed": 1834, + "missed_rescue": 0, + "avoided_harm": 73, + "selected_both_correct": 2616, + "selected_both_wrong": 709 + }, + { + "budget": 0.87, + "threshold": -0.9319414496421814, + "operator": "ge", + "actual_fraction": 0.9558163617535381, + "metric": 0.5600621332412841, + "rescued": 365, + "harmed": 1838, + "missed_rescue": 0, + "avoided_harm": 69, + "selected_both_correct": 2626, + "selected_both_wrong": 709 + }, + { + "budget": 0.88, + "threshold": -0.9340114593505859, + "operator": "ge", + "actual_fraction": 0.958577839143942, + "metric": 0.5591991715567829, + "rescued": 365, + "harmed": 1843, + "missed_rescue": 0, + "avoided_harm": 64, + "selected_both_correct": 2637, + "selected_both_wrong": 709 + }, + { + "budget": 0.89, + "threshold": -0.9363258481025696, + "operator": "ge", + "actual_fraction": 0.9628926475664481, + "metric": 0.5574732481877804, + "rescued": 365, + "harmed": 1853, + "missed_rescue": 0, + "avoided_harm": 54, + "selected_both_correct": 2652, + "selected_both_wrong": 709 + }, + { + "budget": 0.9, + "threshold": -0.9400071501731873, + "operator": "ge", + "actual_fraction": 0.9694511563686572, + "metric": 0.5562651018294787, + "rescued": 365, + "harmed": 1860, + "missed_rescue": 0, + "avoided_harm": 47, + "selected_both_correct": 2683, + "selected_both_wrong": 709 + }, + { + "budget": 0.91, + "threshold": -0.9423198103904724, + "operator": "ge", + "actual_fraction": 0.9734207801173628, + "metric": 0.5550569554711771, + "rescued": 365, + "harmed": 1867, + "missed_rescue": 0, + "avoided_harm": 40, + "selected_both_correct": 2698, + "selected_both_wrong": 710 + }, + { + "budget": 0.92, + "threshold": -0.9443356394767761, + "operator": "ge", + "actual_fraction": 0.9761822575077667, + "metric": 0.5545391784604764, + "rescued": 365, + "harmed": 1870, + "missed_rescue": 0, + "avoided_harm": 37, + "selected_both_correct": 2710, + "selected_both_wrong": 711 + }, + { + "budget": 0.93, + "threshold": -0.9468213319778442, + "operator": "ge", + "actual_fraction": 0.9803244735933725, + "metric": 0.5536762167759751, + "rescued": 365, + "harmed": 1875, + "missed_rescue": 0, + "avoided_harm": 32, + "selected_both_correct": 2729, + "selected_both_wrong": 711 + }, + { + "budget": 0.94, + "threshold": -0.9496435523033142, + "operator": "ge", + "actual_fraction": 0.9837763203313773, + "metric": 0.5526406627545737, + "rescued": 365, + "harmed": 1881, + "missed_rescue": 0, + "avoided_harm": 26, + "selected_both_correct": 2743, + "selected_both_wrong": 711 + }, + { + "budget": 0.95, + "threshold": -0.9529175758361816, + "operator": "ge", + "actual_fraction": 0.9860200207110804, + "metric": 0.5524680704176734, + "rescued": 365, + "harmed": 1882, + "missed_rescue": 0, + "avoided_harm": 25, + "selected_both_correct": 2755, + "selected_both_wrong": 711 + }, + { + "budget": 0.96, + "threshold": -0.9560379981994629, + "operator": "ge", + "actual_fraction": 0.988608905764584, + "metric": 0.5519502934069728, + "rescued": 365, + "harmed": 1885, + "missed_rescue": 0, + "avoided_harm": 22, + "selected_both_correct": 2767, + "selected_both_wrong": 711 + }, + { + "budget": 0.97, + "threshold": -0.9591385126113892, + "operator": "ge", + "actual_fraction": 0.9920607525025888, + "metric": 0.5505695547117708, + "rescued": 365, + "harmed": 1893, + "missed_rescue": 0, + "avoided_harm": 14, + "selected_both_correct": 2779, + "selected_both_wrong": 711 + }, + { + "budget": 0.98, + "threshold": -0.9627243280410767, + "operator": "ge", + "actual_fraction": 0.9946496375560925, + "metric": 0.5498791853641698, + "rescued": 365, + "harmed": 1897, + "missed_rescue": 0, + "avoided_harm": 10, + "selected_both_correct": 2790, + "selected_both_wrong": 711 + }, + { + "budget": 0.99, + "threshold": -0.9702208638191223, + "operator": "ge", + "actual_fraction": 0.9972385226095961, + "metric": 0.5488436313427684, + "rescued": 365, + "harmed": 1903, + "missed_rescue": 0, + "avoided_harm": 4, + "selected_both_correct": 2799, + "selected_both_wrong": 711 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5481532619951675, + "rescued": 365, + "harmed": 1907, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 2811, + "selected_both_wrong": 711 + } + ] + }, + "learned": { + "selected_budget_index": 1, + "validation_area_0_25": 0.9705833333333334, + "validation_point": { + "budget": 0.01, + "threshold": 0.6002578735351562, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9858333333333333, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2, + "avoided_harm": 524, + "selected_both_correct": 11, + "selected_both_wrong": 1 + }, + "test_point": { + "budget": 0.01, + "threshold": 0.6002578735351562, + "operator": "ge", + "actual_fraction": 0.014325163962720055, + "metric": 0.8134276838108389, + "rescued": 5, + "harmed": 10, + "missed_rescue": 360, + "avoided_harm": 1897, + "selected_both_correct": 58, + "selected_both_wrong": 10 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9858333333333333, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2, + "avoided_harm": 524, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 0.6002578735351562, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9858333333333333, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2, + "avoided_harm": 524, + "selected_both_correct": 11, + "selected_both_wrong": 1 + }, + { + "budget": 0.02, + "threshold": -1.3240923881530762, + "operator": "ge", + "actual_fraction": 0.02, + "metric": 0.9841666666666666, + "rescued": 0, + "harmed": 2, + "missed_rescue": 2, + "avoided_harm": 522, + "selected_both_correct": 21, + "selected_both_wrong": 1 + }, + { + "budget": 0.03, + "threshold": -2.1661314964294434, + "operator": "ge", + "actual_fraction": 0.03, + "metric": 0.9816666666666667, + "rescued": 0, + "harmed": 5, + "missed_rescue": 2, + "avoided_harm": 519, + "selected_both_correct": 29, + "selected_both_wrong": 2 + }, + { + "budget": 0.04, + "threshold": -2.9284613132476807, + "operator": "ge", + "actual_fraction": 0.04, + "metric": 0.9816666666666667, + "rescued": 0, + "harmed": 5, + "missed_rescue": 2, + "avoided_harm": 519, + "selected_both_correct": 41, + "selected_both_wrong": 2 + }, + { + "budget": 0.05, + "threshold": -3.3098244667053223, + "operator": "ge", + "actual_fraction": 0.05, + "metric": 0.9816666666666667, + "rescued": 0, + "harmed": 5, + "missed_rescue": 2, + "avoided_harm": 519, + "selected_both_correct": 53, + "selected_both_wrong": 2 + }, + { + "budget": 0.06, + "threshold": -3.7247095108032227, + "operator": "ge", + "actual_fraction": 0.06, + "metric": 0.98, + "rescued": 0, + "harmed": 7, + "missed_rescue": 2, + "avoided_harm": 517, + "selected_both_correct": 62, + "selected_both_wrong": 3 + }, + { + "budget": 0.07, + "threshold": -4.212185859680176, + "operator": "ge", + "actual_fraction": 0.07, + "metric": 0.9775, + "rescued": 0, + "harmed": 10, + "missed_rescue": 2, + "avoided_harm": 514, + "selected_both_correct": 71, + "selected_both_wrong": 3 + }, + { + "budget": 0.08, + "threshold": -4.729281425476074, + "operator": "ge", + "actual_fraction": 0.08, + "metric": 0.975, + "rescued": 0, + "harmed": 13, + "missed_rescue": 2, + "avoided_harm": 511, + "selected_both_correct": 79, + "selected_both_wrong": 4 + }, + { + "budget": 0.09, + "threshold": -5.053535461425781, + "operator": "ge", + "actual_fraction": 0.09, + "metric": 0.9741666666666666, + "rescued": 0, + "harmed": 14, + "missed_rescue": 2, + "avoided_harm": 510, + "selected_both_correct": 90, + "selected_both_wrong": 4 + }, + { + "budget": 0.1, + "threshold": -5.460837364196777, + "operator": "ge", + "actual_fraction": 0.1, + "metric": 0.9741666666666666, + "rescued": 0, + "harmed": 14, + "missed_rescue": 2, + "avoided_harm": 510, + "selected_both_correct": 102, + "selected_both_wrong": 4 + }, + { + "budget": 0.11, + "threshold": -5.743188381195068, + "operator": "ge", + "actual_fraction": 0.11, + "metric": 0.9716666666666667, + "rescued": 0, + "harmed": 17, + "missed_rescue": 2, + "avoided_harm": 507, + "selected_both_correct": 111, + "selected_both_wrong": 4 + }, + { + "budget": 0.12, + "threshold": -6.175364017486572, + "operator": "ge", + "actual_fraction": 0.12, + "metric": 0.9725, + "rescued": 1, + "harmed": 17, + "missed_rescue": 1, + "avoided_harm": 507, + "selected_both_correct": 122, + "selected_both_wrong": 4 + }, + { + "budget": 0.13, + "threshold": -6.506534099578857, + "operator": "ge", + "actual_fraction": 0.13, + "metric": 0.9716666666666667, + "rescued": 1, + "harmed": 18, + "missed_rescue": 1, + "avoided_harm": 506, + "selected_both_correct": 133, + "selected_both_wrong": 4 + }, + { + "budget": 0.14, + "threshold": -6.924992561340332, + "operator": "ge", + "actual_fraction": 0.14, + "metric": 0.9708333333333333, + "rescued": 1, + "harmed": 19, + "missed_rescue": 1, + "avoided_harm": 505, + "selected_both_correct": 144, + "selected_both_wrong": 4 + }, + { + "budget": 0.15, + "threshold": -7.186172962188721, + "operator": "ge", + "actual_fraction": 0.15, + "metric": 0.97, + "rescued": 1, + "harmed": 20, + "missed_rescue": 1, + "avoided_harm": 504, + "selected_both_correct": 155, + "selected_both_wrong": 4 + }, + { + "budget": 0.16, + "threshold": -7.455348014831543, + "operator": "ge", + "actual_fraction": 0.16, + "metric": 0.9691666666666666, + "rescued": 1, + "harmed": 21, + "missed_rescue": 1, + "avoided_harm": 503, + "selected_both_correct": 165, + "selected_both_wrong": 5 + }, + { + "budget": 0.17, + "threshold": -7.729059219360352, + "operator": "ge", + "actual_fraction": 0.17, + "metric": 0.9675, + "rescued": 1, + "harmed": 23, + "missed_rescue": 1, + "avoided_harm": 501, + "selected_both_correct": 175, + "selected_both_wrong": 5 + }, + { + "budget": 0.18, + "threshold": -8.000391006469727, + "operator": "ge", + "actual_fraction": 0.18, + "metric": 0.9641666666666666, + "rescued": 1, + "harmed": 27, + "missed_rescue": 1, + "avoided_harm": 497, + "selected_both_correct": 183, + "selected_both_wrong": 5 + }, + { + "budget": 0.19, + "threshold": -8.232487678527832, + "operator": "ge", + "actual_fraction": 0.19, + "metric": 0.9625, + "rescued": 1, + "harmed": 29, + "missed_rescue": 1, + "avoided_harm": 495, + "selected_both_correct": 192, + "selected_both_wrong": 6 + }, + { + "budget": 0.2, + "threshold": -8.511822700500488, + "operator": "ge", + "actual_fraction": 0.2, + "metric": 0.9616666666666667, + "rescued": 1, + "harmed": 30, + "missed_rescue": 1, + "avoided_harm": 494, + "selected_both_correct": 203, + "selected_both_wrong": 6 + }, + { + "budget": 0.21, + "threshold": -8.755088806152344, + "operator": "ge", + "actual_fraction": 0.21, + "metric": 0.9575, + "rescued": 1, + "harmed": 35, + "missed_rescue": 1, + "avoided_harm": 489, + "selected_both_correct": 210, + "selected_both_wrong": 6 + }, + { + "budget": 0.22, + "threshold": -9.025620460510254, + "operator": "ge", + "actual_fraction": 0.22, + "metric": 0.9533333333333334, + "rescued": 1, + "harmed": 40, + "missed_rescue": 1, + "avoided_harm": 484, + "selected_both_correct": 217, + "selected_both_wrong": 6 + }, + { + "budget": 0.23, + "threshold": -9.279722213745117, + "operator": "ge", + "actual_fraction": 0.23, + "metric": 0.9525, + "rescued": 1, + "harmed": 41, + "missed_rescue": 1, + "avoided_harm": 483, + "selected_both_correct": 227, + "selected_both_wrong": 7 + }, + { + "budget": 0.24, + "threshold": -9.457687377929688, + "operator": "ge", + "actual_fraction": 0.24, + "metric": 0.9483333333333334, + "rescued": 1, + "harmed": 46, + "missed_rescue": 1, + "avoided_harm": 478, + "selected_both_correct": 234, + "selected_both_wrong": 7 + }, + { + "budget": 0.25, + "threshold": -9.643470764160156, + "operator": "ge", + "actual_fraction": 0.25, + "metric": 0.945, + "rescued": 1, + "harmed": 50, + "missed_rescue": 1, + "avoided_harm": 474, + "selected_both_correct": 242, + "selected_both_wrong": 7 + }, + { + "budget": 0.26, + "threshold": -9.806217193603516, + "operator": "ge", + "actual_fraction": 0.26, + "metric": 0.9425, + "rescued": 1, + "harmed": 53, + "missed_rescue": 1, + "avoided_harm": 471, + "selected_both_correct": 250, + "selected_both_wrong": 8 + }, + { + "budget": 0.27, + "threshold": -10.145637512207031, + "operator": "ge", + "actual_fraction": 0.27, + "metric": 0.9383333333333334, + "rescued": 1, + "harmed": 58, + "missed_rescue": 1, + "avoided_harm": 466, + "selected_both_correct": 257, + "selected_both_wrong": 8 + }, + { + "budget": 0.28, + "threshold": -10.341279983520508, + "operator": "ge", + "actual_fraction": 0.28, + "metric": 0.9366666666666666, + "rescued": 1, + "harmed": 60, + "missed_rescue": 1, + "avoided_harm": 464, + "selected_both_correct": 266, + "selected_both_wrong": 9 + }, + { + "budget": 0.29, + "threshold": -10.557610511779785, + "operator": "ge", + "actual_fraction": 0.29, + "metric": 0.935, + "rescued": 1, + "harmed": 62, + "missed_rescue": 1, + "avoided_harm": 462, + "selected_both_correct": 276, + "selected_both_wrong": 9 + }, + { + "budget": 0.3, + "threshold": -10.698262214660645, + "operator": "ge", + "actual_fraction": 0.3, + "metric": 0.935, + "rescued": 1, + "harmed": 62, + "missed_rescue": 1, + "avoided_harm": 462, + "selected_both_correct": 288, + "selected_both_wrong": 9 + }, + { + "budget": 0.31, + "threshold": -10.858185768127441, + "operator": "ge", + "actual_fraction": 0.31, + "metric": 0.9325, + "rescued": 1, + "harmed": 65, + "missed_rescue": 1, + "avoided_harm": 459, + "selected_both_correct": 296, + "selected_both_wrong": 10 + }, + { + "budget": 0.32, + "threshold": -11.096646308898926, + "operator": "ge", + "actual_fraction": 0.32, + "metric": 0.9291666666666667, + "rescued": 1, + "harmed": 69, + "missed_rescue": 1, + "avoided_harm": 455, + "selected_both_correct": 304, + "selected_both_wrong": 10 + }, + { + "budget": 0.33, + "threshold": -11.229066848754883, + "operator": "ge", + "actual_fraction": 0.33, + "metric": 0.9258333333333333, + "rescued": 1, + "harmed": 73, + "missed_rescue": 1, + "avoided_harm": 451, + "selected_both_correct": 312, + "selected_both_wrong": 10 + }, + { + "budget": 0.34, + "threshold": -11.443608283996582, + "operator": "ge", + "actual_fraction": 0.34, + "metric": 0.9233333333333333, + "rescued": 1, + "harmed": 76, + "missed_rescue": 1, + "avoided_harm": 448, + "selected_both_correct": 320, + "selected_both_wrong": 11 + }, + { + "budget": 0.35, + "threshold": -11.589471817016602, + "operator": "ge", + "actual_fraction": 0.35, + "metric": 0.9225, + "rescued": 1, + "harmed": 77, + "missed_rescue": 1, + "avoided_harm": 447, + "selected_both_correct": 331, + "selected_both_wrong": 11 + }, + { + "budget": 0.36, + "threshold": -11.894453048706055, + "operator": "ge", + "actual_fraction": 0.36, + "metric": 0.92, + "rescued": 1, + "harmed": 80, + "missed_rescue": 1, + "avoided_harm": 444, + "selected_both_correct": 340, + "selected_both_wrong": 11 + }, + { + "budget": 0.37, + "threshold": -12.131742477416992, + "operator": "ge", + "actual_fraction": 0.37, + "metric": 0.9166666666666666, + "rescued": 1, + "harmed": 84, + "missed_rescue": 1, + "avoided_harm": 440, + "selected_both_correct": 348, + "selected_both_wrong": 11 + }, + { + "budget": 0.38, + "threshold": -12.303675651550293, + "operator": "ge", + "actual_fraction": 0.38, + "metric": 0.915, + "rescued": 1, + "harmed": 86, + "missed_rescue": 1, + "avoided_harm": 438, + "selected_both_correct": 358, + "selected_both_wrong": 11 + }, + { + "budget": 0.39, + "threshold": -12.509767532348633, + "operator": "ge", + "actual_fraction": 0.39, + "metric": 0.9075, + "rescued": 1, + "harmed": 95, + "missed_rescue": 1, + "avoided_harm": 429, + "selected_both_correct": 360, + "selected_both_wrong": 12 + }, + { + "budget": 0.4, + "threshold": -12.680671691894531, + "operator": "ge", + "actual_fraction": 0.4, + "metric": 0.9041666666666667, + "rescued": 1, + "harmed": 99, + "missed_rescue": 1, + "avoided_harm": 425, + "selected_both_correct": 368, + "selected_both_wrong": 12 + }, + { + "budget": 0.41, + "threshold": -12.899209022521973, + "operator": "ge", + "actual_fraction": 0.4091666666666667, + "metric": 0.9016666666666666, + "rescued": 1, + "harmed": 102, + "missed_rescue": 1, + "avoided_harm": 422, + "selected_both_correct": 376, + "selected_both_wrong": 12 + }, + { + "budget": 0.42, + "threshold": -13.105607986450195, + "operator": "ge", + "actual_fraction": 0.42, + "metric": 0.8975, + "rescued": 1, + "harmed": 107, + "missed_rescue": 1, + "avoided_harm": 417, + "selected_both_correct": 384, + "selected_both_wrong": 12 + }, + { + "budget": 0.43, + "threshold": -13.328797340393066, + "operator": "ge", + "actual_fraction": 0.43, + "metric": 0.895, + "rescued": 1, + "harmed": 110, + "missed_rescue": 1, + "avoided_harm": 414, + "selected_both_correct": 393, + "selected_both_wrong": 12 + }, + { + "budget": 0.44, + "threshold": -13.622771263122559, + "operator": "ge", + "actual_fraction": 0.44, + "metric": 0.89, + "rescued": 1, + "harmed": 116, + "missed_rescue": 1, + "avoided_harm": 408, + "selected_both_correct": 399, + "selected_both_wrong": 12 + }, + { + "budget": 0.45, + "threshold": -13.815265655517578, + "operator": "ge", + "actual_fraction": 0.45, + "metric": 0.8858333333333334, + "rescued": 1, + "harmed": 121, + "missed_rescue": 1, + "avoided_harm": 403, + "selected_both_correct": 406, + "selected_both_wrong": 12 + }, + { + "budget": 0.46, + "threshold": -13.881027221679688, + "operator": "ge", + "actual_fraction": 0.46, + "metric": 0.88, + "rescued": 1, + "harmed": 128, + "missed_rescue": 1, + "avoided_harm": 396, + "selected_both_correct": 410, + "selected_both_wrong": 13 + }, + { + "budget": 0.47, + "threshold": -14.048774719238281, + "operator": "ge", + "actual_fraction": 0.47, + "metric": 0.8766666666666667, + "rescued": 1, + "harmed": 132, + "missed_rescue": 1, + "avoided_harm": 392, + "selected_both_correct": 418, + "selected_both_wrong": 13 + }, + { + "budget": 0.48, + "threshold": -14.219049453735352, + "operator": "ge", + "actual_fraction": 0.48, + "metric": 0.8741666666666666, + "rescued": 1, + "harmed": 135, + "missed_rescue": 1, + "avoided_harm": 389, + "selected_both_correct": 427, + "selected_both_wrong": 13 + }, + { + "budget": 0.49, + "threshold": -14.446796417236328, + "operator": "ge", + "actual_fraction": 0.49, + "metric": 0.8708333333333333, + "rescued": 1, + "harmed": 139, + "missed_rescue": 1, + "avoided_harm": 385, + "selected_both_correct": 435, + "selected_both_wrong": 13 + }, + { + "budget": 0.5, + "threshold": -14.658285140991211, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.865, + "rescued": 1, + "harmed": 146, + "missed_rescue": 1, + "avoided_harm": 378, + "selected_both_correct": 440, + "selected_both_wrong": 13 + }, + { + "budget": 0.51, + "threshold": -14.863032341003418, + "operator": "ge", + "actual_fraction": 0.51, + "metric": 0.86, + "rescued": 1, + "harmed": 152, + "missed_rescue": 1, + "avoided_harm": 372, + "selected_both_correct": 446, + "selected_both_wrong": 13 + }, + { + "budget": 0.52, + "threshold": -15.09691333770752, + "operator": "ge", + "actual_fraction": 0.52, + "metric": 0.8558333333333333, + "rescued": 1, + "harmed": 157, + "missed_rescue": 1, + "avoided_harm": 367, + "selected_both_correct": 453, + "selected_both_wrong": 13 + }, + { + "budget": 0.53, + "threshold": -15.207231521606445, + "operator": "ge", + "actual_fraction": 0.53, + "metric": 0.8491666666666666, + "rescued": 1, + "harmed": 165, + "missed_rescue": 1, + "avoided_harm": 359, + "selected_both_correct": 457, + "selected_both_wrong": 13 + }, + { + "budget": 0.54, + "threshold": -15.392038345336914, + "operator": "ge", + "actual_fraction": 0.54, + "metric": 0.8441666666666666, + "rescued": 2, + "harmed": 172, + "missed_rescue": 0, + "avoided_harm": 352, + "selected_both_correct": 461, + "selected_both_wrong": 13 + }, + { + "budget": 0.55, + "threshold": -15.607501983642578, + "operator": "ge", + "actual_fraction": 0.55, + "metric": 0.8383333333333334, + "rescued": 2, + "harmed": 179, + "missed_rescue": 0, + "avoided_harm": 345, + "selected_both_correct": 466, + "selected_both_wrong": 13 + }, + { + "budget": 0.56, + "threshold": -15.75269889831543, + "operator": "ge", + "actual_fraction": 0.56, + "metric": 0.8316666666666667, + "rescued": 2, + "harmed": 187, + "missed_rescue": 0, + "avoided_harm": 337, + "selected_both_correct": 469, + "selected_both_wrong": 14 + }, + { + "budget": 0.57, + "threshold": -15.911096572875977, + "operator": "ge", + "actual_fraction": 0.5691666666666667, + "metric": 0.8283333333333334, + "rescued": 2, + "harmed": 191, + "missed_rescue": 0, + "avoided_harm": 333, + "selected_both_correct": 476, + "selected_both_wrong": 14 + }, + { + "budget": 0.58, + "threshold": -16.042922973632812, + "operator": "ge", + "actual_fraction": 0.58, + "metric": 0.8225, + "rescued": 2, + "harmed": 198, + "missed_rescue": 0, + "avoided_harm": 326, + "selected_both_correct": 482, + "selected_both_wrong": 14 + }, + { + "budget": 0.59, + "threshold": -16.200180053710938, + "operator": "ge", + "actual_fraction": 0.59, + "metric": 0.8166666666666667, + "rescued": 2, + "harmed": 205, + "missed_rescue": 0, + "avoided_harm": 319, + "selected_both_correct": 487, + "selected_both_wrong": 14 + }, + { + "budget": 0.6, + "threshold": -16.35162925720215, + "operator": "ge", + "actual_fraction": 0.6, + "metric": 0.8116666666666666, + "rescued": 2, + "harmed": 211, + "missed_rescue": 0, + "avoided_harm": 313, + "selected_both_correct": 493, + "selected_both_wrong": 14 + }, + { + "budget": 0.61, + "threshold": -16.515636444091797, + "operator": "ge", + "actual_fraction": 0.61, + "metric": 0.8066666666666666, + "rescued": 2, + "harmed": 217, + "missed_rescue": 0, + "avoided_harm": 307, + "selected_both_correct": 499, + "selected_both_wrong": 14 + }, + { + "budget": 0.62, + "threshold": -16.80984878540039, + "operator": "ge", + "actual_fraction": 0.62, + "metric": 0.805, + "rescued": 2, + "harmed": 219, + "missed_rescue": 0, + "avoided_harm": 305, + "selected_both_correct": 509, + "selected_both_wrong": 14 + }, + { + "budget": 0.63, + "threshold": -17.124507904052734, + "operator": "ge", + "actual_fraction": 0.63, + "metric": 0.8008333333333333, + "rescued": 2, + "harmed": 224, + "missed_rescue": 0, + "avoided_harm": 300, + "selected_both_correct": 516, + "selected_both_wrong": 14 + }, + { + "budget": 0.64, + "threshold": -17.39409637451172, + "operator": "ge", + "actual_fraction": 0.64, + "metric": 0.795, + "rescued": 2, + "harmed": 231, + "missed_rescue": 0, + "avoided_harm": 293, + "selected_both_correct": 521, + "selected_both_wrong": 14 + }, + { + "budget": 0.65, + "threshold": -17.61226463317871, + "operator": "ge", + "actual_fraction": 0.65, + "metric": 0.7875, + "rescued": 2, + "harmed": 240, + "missed_rescue": 0, + "avoided_harm": 284, + "selected_both_correct": 524, + "selected_both_wrong": 14 + }, + { + "budget": 0.66, + "threshold": -17.802616119384766, + "operator": "ge", + "actual_fraction": 0.66, + "metric": 0.785, + "rescued": 2, + "harmed": 243, + "missed_rescue": 0, + "avoided_harm": 281, + "selected_both_correct": 533, + "selected_both_wrong": 14 + }, + { + "budget": 0.67, + "threshold": -17.987302780151367, + "operator": "ge", + "actual_fraction": 0.67, + "metric": 0.78, + "rescued": 2, + "harmed": 249, + "missed_rescue": 0, + "avoided_harm": 275, + "selected_both_correct": 539, + "selected_both_wrong": 14 + }, + { + "budget": 0.68, + "threshold": -18.25859832763672, + "operator": "ge", + "actual_fraction": 0.68, + "metric": 0.7733333333333333, + "rescued": 2, + "harmed": 257, + "missed_rescue": 0, + "avoided_harm": 267, + "selected_both_correct": 543, + "selected_both_wrong": 14 + }, + { + "budget": 0.69, + "threshold": -18.391502380371094, + "operator": "ge", + "actual_fraction": 0.6891666666666667, + "metric": 0.77, + "rescued": 2, + "harmed": 261, + "missed_rescue": 0, + "avoided_harm": 263, + "selected_both_correct": 550, + "selected_both_wrong": 14 + }, + { + "budget": 0.7, + "threshold": -18.60244369506836, + "operator": "ge", + "actual_fraction": 0.7, + "metric": 0.7633333333333333, + "rescued": 2, + "harmed": 269, + "missed_rescue": 0, + "avoided_harm": 255, + "selected_both_correct": 555, + "selected_both_wrong": 14 + }, + { + "budget": 0.71, + "threshold": -18.753427505493164, + "operator": "ge", + "actual_fraction": 0.71, + "metric": 0.7583333333333333, + "rescued": 2, + "harmed": 275, + "missed_rescue": 0, + "avoided_harm": 249, + "selected_both_correct": 561, + "selected_both_wrong": 14 + }, + { + "budget": 0.72, + "threshold": -18.970993041992188, + "operator": "ge", + "actual_fraction": 0.72, + "metric": 0.7525, + "rescued": 2, + "harmed": 282, + "missed_rescue": 0, + "avoided_harm": 242, + "selected_both_correct": 566, + "selected_both_wrong": 14 + }, + { + "budget": 0.73, + "threshold": -19.116729736328125, + "operator": "ge", + "actual_fraction": 0.73, + "metric": 0.7491666666666666, + "rescued": 2, + "harmed": 286, + "missed_rescue": 0, + "avoided_harm": 238, + "selected_both_correct": 574, + "selected_both_wrong": 14 + }, + { + "budget": 0.74, + "threshold": -19.298297882080078, + "operator": "ge", + "actual_fraction": 0.74, + "metric": 0.7425, + "rescued": 2, + "harmed": 294, + "missed_rescue": 0, + "avoided_harm": 230, + "selected_both_correct": 578, + "selected_both_wrong": 14 + }, + { + "budget": 0.75, + "threshold": -19.566133499145508, + "operator": "ge", + "actual_fraction": 0.75, + "metric": 0.7358333333333333, + "rescued": 2, + "harmed": 302, + "missed_rescue": 0, + "avoided_harm": 222, + "selected_both_correct": 582, + "selected_both_wrong": 14 + }, + { + "budget": 0.76, + "threshold": -19.99453353881836, + "operator": "ge", + "actual_fraction": 0.76, + "metric": 0.73, + "rescued": 2, + "harmed": 309, + "missed_rescue": 0, + "avoided_harm": 215, + "selected_both_correct": 587, + "selected_both_wrong": 14 + }, + { + "budget": 0.77, + "threshold": -20.128440856933594, + "operator": "ge", + "actual_fraction": 0.77, + "metric": 0.7258333333333333, + "rescued": 2, + "harmed": 314, + "missed_rescue": 0, + "avoided_harm": 210, + "selected_both_correct": 593, + "selected_both_wrong": 15 + }, + { + "budget": 0.78, + "threshold": -20.354900360107422, + "operator": "ge", + "actual_fraction": 0.78, + "metric": 0.7166666666666667, + "rescued": 2, + "harmed": 325, + "missed_rescue": 0, + "avoided_harm": 199, + "selected_both_correct": 594, + "selected_both_wrong": 15 + }, + { + "budget": 0.79, + "threshold": -20.70088005065918, + "operator": "ge", + "actual_fraction": 0.79, + "metric": 0.7091666666666666, + "rescued": 2, + "harmed": 334, + "missed_rescue": 0, + "avoided_harm": 190, + "selected_both_correct": 597, + "selected_both_wrong": 15 + }, + { + "budget": 0.8, + "threshold": -21.008874893188477, + "operator": "ge", + "actual_fraction": 0.8, + "metric": 0.7033333333333334, + "rescued": 2, + "harmed": 341, + "missed_rescue": 0, + "avoided_harm": 183, + "selected_both_correct": 602, + "selected_both_wrong": 15 + }, + { + "budget": 0.81, + "threshold": -21.276046752929688, + "operator": "ge", + "actual_fraction": 0.81, + "metric": 0.6983333333333334, + "rescued": 2, + "harmed": 347, + "missed_rescue": 0, + "avoided_harm": 177, + "selected_both_correct": 608, + "selected_both_wrong": 15 + }, + { + "budget": 0.82, + "threshold": -21.52347183227539, + "operator": "ge", + "actual_fraction": 0.8191666666666667, + "metric": 0.6941666666666667, + "rescued": 2, + "harmed": 352, + "missed_rescue": 0, + "avoided_harm": 172, + "selected_both_correct": 614, + "selected_both_wrong": 15 + }, + { + "budget": 0.83, + "threshold": -21.878074645996094, + "operator": "ge", + "actual_fraction": 0.83, + "metric": 0.6883333333333334, + "rescued": 2, + "harmed": 359, + "missed_rescue": 0, + "avoided_harm": 165, + "selected_both_correct": 620, + "selected_both_wrong": 15 + }, + { + "budget": 0.84, + "threshold": -22.360363006591797, + "operator": "ge", + "actual_fraction": 0.84, + "metric": 0.6841666666666667, + "rescued": 2, + "harmed": 364, + "missed_rescue": 0, + "avoided_harm": 160, + "selected_both_correct": 627, + "selected_both_wrong": 15 + }, + { + "budget": 0.85, + "threshold": -22.77996826171875, + "operator": "ge", + "actual_fraction": 0.85, + "metric": 0.6775, + "rescued": 2, + "harmed": 372, + "missed_rescue": 0, + "avoided_harm": 152, + "selected_both_correct": 631, + "selected_both_wrong": 15 + }, + { + "budget": 0.86, + "threshold": -23.05141830444336, + "operator": "ge", + "actual_fraction": 0.86, + "metric": 0.67, + "rescued": 2, + "harmed": 381, + "missed_rescue": 0, + "avoided_harm": 143, + "selected_both_correct": 634, + "selected_both_wrong": 15 + }, + { + "budget": 0.87, + "threshold": -23.436256408691406, + "operator": "ge", + "actual_fraction": 0.87, + "metric": 0.6633333333333333, + "rescued": 2, + "harmed": 389, + "missed_rescue": 0, + "avoided_harm": 135, + "selected_both_correct": 638, + "selected_both_wrong": 15 + }, + { + "budget": 0.88, + "threshold": -23.772342681884766, + "operator": "ge", + "actual_fraction": 0.88, + "metric": 0.6566666666666666, + "rescued": 2, + "harmed": 397, + "missed_rescue": 0, + "avoided_harm": 127, + "selected_both_correct": 642, + "selected_both_wrong": 15 + }, + { + "budget": 0.89, + "threshold": -24.192012786865234, + "operator": "ge", + "actual_fraction": 0.89, + "metric": 0.6491666666666667, + "rescued": 2, + "harmed": 406, + "missed_rescue": 0, + "avoided_harm": 118, + "selected_both_correct": 645, + "selected_both_wrong": 15 + }, + { + "budget": 0.9, + "threshold": -24.707534790039062, + "operator": "ge", + "actual_fraction": 0.9, + "metric": 0.6425, + "rescued": 2, + "harmed": 414, + "missed_rescue": 0, + "avoided_harm": 110, + "selected_both_correct": 649, + "selected_both_wrong": 15 + }, + { + "budget": 0.91, + "threshold": -25.141036987304688, + "operator": "ge", + "actual_fraction": 0.91, + "metric": 0.6358333333333334, + "rescued": 2, + "harmed": 422, + "missed_rescue": 0, + "avoided_harm": 102, + "selected_both_correct": 653, + "selected_both_wrong": 15 + }, + { + "budget": 0.92, + "threshold": -25.627573013305664, + "operator": "ge", + "actual_fraction": 0.92, + "metric": 0.6266666666666667, + "rescued": 2, + "harmed": 433, + "missed_rescue": 0, + "avoided_harm": 91, + "selected_both_correct": 654, + "selected_both_wrong": 15 + }, + { + "budget": 0.93, + "threshold": -26.255355834960938, + "operator": "ge", + "actual_fraction": 0.93, + "metric": 0.6175, + "rescued": 2, + "harmed": 444, + "missed_rescue": 0, + "avoided_harm": 80, + "selected_both_correct": 655, + "selected_both_wrong": 15 + }, + { + "budget": 0.94, + "threshold": -26.890518188476562, + "operator": "ge", + "actual_fraction": 0.94, + "metric": 0.6075, + "rescued": 2, + "harmed": 456, + "missed_rescue": 0, + "avoided_harm": 68, + "selected_both_correct": 655, + "selected_both_wrong": 15 + }, + { + "budget": 0.95, + "threshold": -27.40430450439453, + "operator": "ge", + "actual_fraction": 0.95, + "metric": 0.6, + "rescued": 2, + "harmed": 465, + "missed_rescue": 0, + "avoided_harm": 59, + "selected_both_correct": 658, + "selected_both_wrong": 15 + }, + { + "budget": 0.96, + "threshold": -27.9757022857666, + "operator": "ge", + "actual_fraction": 0.96, + "metric": 0.59, + "rescued": 2, + "harmed": 477, + "missed_rescue": 0, + "avoided_harm": 47, + "selected_both_correct": 658, + "selected_both_wrong": 15 + }, + { + "budget": 0.97, + "threshold": -29.13153839111328, + "operator": "ge", + "actual_fraction": 0.97, + "metric": 0.58, + "rescued": 2, + "harmed": 489, + "missed_rescue": 0, + "avoided_harm": 35, + "selected_both_correct": 658, + "selected_both_wrong": 15 + }, + { + "budget": 0.98, + "threshold": -30.387767791748047, + "operator": "ge", + "actual_fraction": 0.98, + "metric": 0.57, + "rescued": 2, + "harmed": 501, + "missed_rescue": 0, + "avoided_harm": 23, + "selected_both_correct": 658, + "selected_both_wrong": 15 + }, + { + "budget": 0.99, + "threshold": -32.89930725097656, + "operator": "ge", + "actual_fraction": 0.99, + "metric": 0.5608333333333333, + "rescued": 2, + "harmed": 512, + "missed_rescue": 0, + "avoided_harm": 12, + "selected_both_correct": 659, + "selected_both_wrong": 15 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5508333333333333, + "rescued": 2, + "harmed": 524, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 659, + "selected_both_wrong": 15 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.81429064549534, + "rescued": 0, + "harmed": 0, + "missed_rescue": 365, + "avoided_harm": 1907, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 0.6002578735351562, + "operator": "ge", + "actual_fraction": 0.014325163962720055, + "metric": 0.8134276838108389, + "rescued": 5, + "harmed": 10, + "missed_rescue": 360, + "avoided_harm": 1897, + "selected_both_correct": 58, + "selected_both_wrong": 10 + }, + { + "budget": 0.02, + "threshold": -1.3240923881530762, + "operator": "ge", + "actual_fraction": 0.029340697273041078, + "metric": 0.8123921297894373, + "rescued": 11, + "harmed": 22, + "missed_rescue": 354, + "avoided_harm": 1885, + "selected_both_correct": 117, + "selected_both_wrong": 20 + }, + { + "budget": 0.03, + "threshold": -2.1661314964294434, + "operator": "ge", + "actual_fraction": 0.04055919917155678, + "metric": 0.8127373144632378, + "rescued": 20, + "harmed": 29, + "missed_rescue": 345, + "avoided_harm": 1878, + "selected_both_correct": 162, + "selected_both_wrong": 24 + }, + { + "budget": 0.04, + "threshold": -2.9284613132476807, + "operator": "ge", + "actual_fraction": 0.05350362443907491, + "metric": 0.8132550914739386, + "rescued": 32, + "harmed": 38, + "missed_rescue": 333, + "avoided_harm": 1869, + "selected_both_correct": 203, + "selected_both_wrong": 37 + }, + { + "budget": 0.05, + "threshold": -3.3098244667053223, + "operator": "ge", + "actual_fraction": 0.06144287193648602, + "metric": 0.8127373144632378, + "rescued": 36, + "harmed": 45, + "missed_rescue": 329, + "avoided_harm": 1862, + "selected_both_correct": 232, + "selected_both_wrong": 43 + }, + { + "budget": 0.06, + "threshold": -3.7247095108032227, + "operator": "ge", + "actual_fraction": 0.07041767345529859, + "metric": 0.8117017604418364, + "rescued": 37, + "harmed": 52, + "missed_rescue": 328, + "avoided_harm": 1855, + "selected_both_correct": 270, + "selected_both_wrong": 49 + }, + { + "budget": 0.07, + "threshold": -4.212185859680176, + "operator": "ge", + "actual_fraction": 0.08094580600621333, + "metric": 0.8115291681049361, + "rescued": 40, + "harmed": 56, + "missed_rescue": 325, + "avoided_harm": 1851, + "selected_both_correct": 313, + "selected_both_wrong": 60 + }, + { + "budget": 0.08, + "threshold": -4.729281425476074, + "operator": "ge", + "actual_fraction": 0.09216430790472903, + "metric": 0.8104936140835347, + "rescued": 45, + "harmed": 67, + "missed_rescue": 320, + "avoided_harm": 1840, + "selected_both_correct": 351, + "selected_both_wrong": 71 + }, + { + "budget": 0.09, + "threshold": -5.053535461425781, + "operator": "ge", + "actual_fraction": 0.09975837072833967, + "metric": 0.809285467725233, + "rescued": 46, + "harmed": 75, + "missed_rescue": 319, + "avoided_harm": 1832, + "selected_both_correct": 382, + "selected_both_wrong": 75 + }, + { + "budget": 0.1, + "threshold": -5.460837364196777, + "operator": "ge", + "actual_fraction": 0.10942354159475319, + "metric": 0.8087676907145323, + "rescued": 50, + "harmed": 82, + "missed_rescue": 315, + "avoided_harm": 1825, + "selected_both_correct": 421, + "selected_both_wrong": 81 + }, + { + "budget": 0.11, + "threshold": -5.743188381195068, + "operator": "ge", + "actual_fraction": 0.11788056610286503, + "metric": 0.8075595443562306, + "rescued": 54, + "harmed": 93, + "missed_rescue": 311, + "avoided_harm": 1814, + "selected_both_correct": 449, + "selected_both_wrong": 87 + }, + { + "budget": 0.12, + "threshold": -6.175364017486572, + "operator": "ge", + "actual_fraction": 0.13047980669658268, + "metric": 0.80721435968243, + "rescued": 60, + "harmed": 101, + "missed_rescue": 305, + "avoided_harm": 1806, + "selected_both_correct": 505, + "selected_both_wrong": 90 + }, + { + "budget": 0.13, + "threshold": -6.506534099578857, + "operator": "ge", + "actual_fraction": 0.14290645495340007, + "metric": 0.8063513979979289, + "rescued": 66, + "harmed": 112, + "missed_rescue": 299, + "avoided_harm": 1795, + "selected_both_correct": 553, + "selected_both_wrong": 97 + }, + { + "budget": 0.14, + "threshold": -6.924992561340332, + "operator": "ge", + "actual_fraction": 0.1563686572316189, + "metric": 0.804280289955126, + "rescued": 70, + "harmed": 128, + "missed_rescue": 295, + "avoided_harm": 1779, + "selected_both_correct": 600, + "selected_both_wrong": 108 + }, + { + "budget": 0.15, + "threshold": -7.186172962188721, + "operator": "ge", + "actual_fraction": 0.16637901277183292, + "metric": 0.8039351052813255, + "rescued": 75, + "harmed": 135, + "missed_rescue": 290, + "avoided_harm": 1772, + "selected_both_correct": 642, + "selected_both_wrong": 112 + }, + { + "budget": 0.16, + "threshold": -7.455348014831543, + "operator": "ge", + "actual_fraction": 0.17846047635484985, + "metric": 0.8023817742492233, + "rescued": 78, + "harmed": 147, + "missed_rescue": 287, + "avoided_harm": 1760, + "selected_both_correct": 688, + "selected_both_wrong": 121 + }, + { + "budget": 0.17, + "threshold": -7.729059219360352, + "operator": "ge", + "actual_fraction": 0.18916120124266483, + "metric": 0.8011736278909216, + "rescued": 84, + "harmed": 160, + "missed_rescue": 281, + "avoided_harm": 1747, + "selected_both_correct": 725, + "selected_both_wrong": 127 + }, + { + "budget": 0.18, + "threshold": -8.000391006469727, + "operator": "ge", + "actual_fraction": 0.20176044183638248, + "metric": 0.7994477045219193, + "rescued": 86, + "harmed": 172, + "missed_rescue": 279, + "avoided_harm": 1735, + "selected_both_correct": 778, + "selected_both_wrong": 133 + }, + { + "budget": 0.19, + "threshold": -8.232487678527832, + "operator": "ge", + "actual_fraction": 0.211425612702796, + "metric": 0.7984121505005177, + "rescued": 96, + "harmed": 188, + "missed_rescue": 269, + "avoided_harm": 1719, + "selected_both_correct": 803, + "selected_both_wrong": 138 + }, + { + "budget": 0.2, + "threshold": -8.511822700500488, + "operator": "ge", + "actual_fraction": 0.2222989299275112, + "metric": 0.7975491888160166, + "rescued": 98, + "harmed": 195, + "missed_rescue": 267, + "avoided_harm": 1712, + "selected_both_correct": 848, + "selected_both_wrong": 147 + }, + { + "budget": 0.21, + "threshold": -8.755088806152344, + "operator": "ge", + "actual_fraction": 0.2326544701415257, + "metric": 0.7970314118053159, + "rescued": 103, + "harmed": 203, + "missed_rescue": 262, + "avoided_harm": 1704, + "selected_both_correct": 887, + "selected_both_wrong": 155 + }, + { + "budget": 0.22, + "threshold": -9.025620460510254, + "operator": "ge", + "actual_fraction": 0.24577148774594407, + "metric": 0.7954780807732137, + "rescued": 109, + "harmed": 218, + "missed_rescue": 256, + "avoided_harm": 1689, + "selected_both_correct": 929, + "selected_both_wrong": 168 + }, + { + "budget": 0.23, + "threshold": -9.279722213745117, + "operator": "ge", + "actual_fraction": 0.25509147393855713, + "metric": 0.7937521574042112, + "rescued": 116, + "harmed": 235, + "missed_rescue": 249, + "avoided_harm": 1672, + "selected_both_correct": 952, + "selected_both_wrong": 175 + }, + { + "budget": 0.24, + "threshold": -9.457687377929688, + "operator": "ge", + "actual_fraction": 0.26354849844666894, + "metric": 0.793579565067311, + "rescued": 120, + "harmed": 240, + "missed_rescue": 245, + "avoided_harm": 1667, + "selected_both_correct": 986, + "selected_both_wrong": 181 + }, + { + "budget": 0.25, + "threshold": -9.643470764160156, + "operator": "ge", + "actual_fraction": 0.2714877459440801, + "metric": 0.7928891957197101, + "rescued": 126, + "harmed": 250, + "missed_rescue": 239, + "avoided_harm": 1657, + "selected_both_correct": 1013, + "selected_both_wrong": 184 + }, + { + "budget": 0.26, + "threshold": -9.806217193603516, + "operator": "ge", + "actual_fraction": 0.27804625474628925, + "metric": 0.7923714187090093, + "rescued": 127, + "harmed": 254, + "missed_rescue": 238, + "avoided_harm": 1653, + "selected_both_correct": 1042, + "selected_both_wrong": 188 + }, + { + "budget": 0.27, + "threshold": -10.145637512207031, + "operator": "ge", + "actual_fraction": 0.2961684501208146, + "metric": 0.7896099413186054, + "rescued": 135, + "harmed": 278, + "missed_rescue": 230, + "avoided_harm": 1629, + "selected_both_correct": 1106, + "selected_both_wrong": 197 + }, + { + "budget": 0.28, + "threshold": -10.341279983520508, + "operator": "ge", + "actual_fraction": 0.3063513979979289, + "metric": 0.7880566102865033, + "rescued": 139, + "harmed": 291, + "missed_rescue": 226, + "avoided_harm": 1616, + "selected_both_correct": 1141, + "selected_both_wrong": 204 + }, + { + "budget": 0.29, + "threshold": -10.557610511779785, + "operator": "ge", + "actual_fraction": 0.31601656886434243, + "metric": 0.7863306869175009, + "rescued": 141, + "harmed": 303, + "missed_rescue": 224, + "avoided_harm": 1604, + "selected_both_correct": 1173, + "selected_both_wrong": 214 + }, + { + "budget": 0.3, + "threshold": -10.698262214660645, + "operator": "ge", + "actual_fraction": 0.32430100103555404, + "metric": 0.7851225405591992, + "rescued": 143, + "harmed": 312, + "missed_rescue": 222, + "avoided_harm": 1595, + "selected_both_correct": 1206, + "selected_both_wrong": 218 + }, + { + "budget": 0.31, + "threshold": -10.858185768127441, + "operator": "ge", + "actual_fraction": 0.3334483948912668, + "metric": 0.783569209527097, + "rescued": 145, + "harmed": 323, + "missed_rescue": 220, + "avoided_harm": 1584, + "selected_both_correct": 1239, + "selected_both_wrong": 225 + }, + { + "budget": 0.32, + "threshold": -11.096646308898926, + "operator": "ge", + "actual_fraction": 0.3465654124956852, + "metric": 0.7814981014842941, + "rescued": 151, + "harmed": 341, + "missed_rescue": 214, + "avoided_harm": 1566, + "selected_both_correct": 1282, + "selected_both_wrong": 234 + }, + { + "budget": 0.33, + "threshold": -11.229066848754883, + "operator": "ge", + "actual_fraction": 0.35226095961339315, + "metric": 0.7808077321366931, + "rescued": 153, + "harmed": 347, + "missed_rescue": 212, + "avoided_harm": 1560, + "selected_both_correct": 1301, + "selected_both_wrong": 240 + }, + { + "budget": 0.34, + "threshold": -11.443608283996582, + "operator": "ge", + "actual_fraction": 0.3639972385226096, + "metric": 0.7806351397997929, + "rescued": 163, + "harmed": 358, + "missed_rescue": 202, + "avoided_harm": 1549, + "selected_both_correct": 1339, + "selected_both_wrong": 249 + }, + { + "budget": 0.35, + "threshold": -11.589471817016602, + "operator": "ge", + "actual_fraction": 0.3738350017259234, + "metric": 0.7794269934414912, + "rescued": 167, + "harmed": 369, + "missed_rescue": 198, + "avoided_harm": 1538, + "selected_both_correct": 1372, + "selected_both_wrong": 258 + }, + { + "budget": 0.36, + "threshold": -11.894453048706055, + "operator": "ge", + "actual_fraction": 0.3905764584052468, + "metric": 0.7754573696927857, + "rescued": 172, + "harmed": 397, + "missed_rescue": 193, + "avoided_harm": 1510, + "selected_both_correct": 1418, + "selected_both_wrong": 276 + }, + { + "budget": 0.37, + "threshold": -12.131742477416992, + "operator": "ge", + "actual_fraction": 0.40231273731446326, + "metric": 0.7725232999654815, + "rescued": 176, + "harmed": 418, + "missed_rescue": 189, + "avoided_harm": 1489, + "selected_both_correct": 1448, + "selected_both_wrong": 289 + }, + { + "budget": 0.38, + "threshold": -12.303675651550293, + "operator": "ge", + "actual_fraction": 0.41197790818087676, + "metric": 0.7716603382809804, + "rescued": 181, + "harmed": 428, + "missed_rescue": 184, + "avoided_harm": 1479, + "selected_both_correct": 1481, + "selected_both_wrong": 297 + }, + { + "budget": 0.39, + "threshold": -12.509767532348633, + "operator": "ge", + "actual_fraction": 0.42267863306869174, + "metric": 0.7699344149119779, + "rescued": 186, + "harmed": 443, + "missed_rescue": 179, + "avoided_harm": 1464, + "selected_both_correct": 1517, + "selected_both_wrong": 303 + }, + { + "budget": 0.4, + "threshold": -12.680671691894531, + "operator": "ge", + "actual_fraction": 0.43148084225060407, + "metric": 0.768553676216776, + "rescued": 192, + "harmed": 457, + "missed_rescue": 173, + "avoided_harm": 1450, + "selected_both_correct": 1544, + "selected_both_wrong": 307 + }, + { + "budget": 0.41, + "threshold": -12.899209022521973, + "operator": "ge", + "actual_fraction": 0.44287193648602, + "metric": 0.767863306869175, + "rescued": 199, + "harmed": 468, + "missed_rescue": 166, + "avoided_harm": 1439, + "selected_both_correct": 1580, + "selected_both_wrong": 319 + }, + { + "budget": 0.42, + "threshold": -13.105607986450195, + "operator": "ge", + "actual_fraction": 0.45460821539523644, + "metric": 0.7654470141525717, + "rescued": 202, + "harmed": 485, + "missed_rescue": 163, + "avoided_harm": 1422, + "selected_both_correct": 1619, + "selected_both_wrong": 328 + }, + { + "budget": 0.43, + "threshold": -13.328797340393066, + "operator": "ge", + "actual_fraction": 0.4689333793579565, + "metric": 0.7630307214359683, + "rescued": 208, + "harmed": 505, + "missed_rescue": 157, + "avoided_harm": 1402, + "selected_both_correct": 1667, + "selected_both_wrong": 337 + }, + { + "budget": 0.44, + "threshold": -13.622771263122559, + "operator": "ge", + "actual_fraction": 0.48394891266827755, + "metric": 0.7619951674145667, + "rescued": 215, + "harmed": 518, + "missed_rescue": 150, + "avoided_harm": 1389, + "selected_both_correct": 1727, + "selected_both_wrong": 344 + }, + { + "budget": 0.45, + "threshold": -13.815265655517578, + "operator": "ge", + "actual_fraction": 0.49309630652399034, + "metric": 0.7600966517086641, + "rescued": 218, + "harmed": 532, + "missed_rescue": 147, + "avoided_harm": 1375, + "selected_both_correct": 1751, + "selected_both_wrong": 356 + }, + { + "budget": 0.46, + "threshold": -13.881027221679688, + "operator": "ge", + "actual_fraction": 0.49654815326199514, + "metric": 0.759233690024163, + "rescued": 219, + "harmed": 538, + "missed_rescue": 146, + "avoided_harm": 1369, + "selected_both_correct": 1762, + "selected_both_wrong": 358 + }, + { + "budget": 0.47, + "threshold": -14.048774719238281, + "operator": "ge", + "actual_fraction": 0.5074214704867104, + "metric": 0.7568173973075596, + "rescued": 223, + "harmed": 556, + "missed_rescue": 142, + "avoided_harm": 1351, + "selected_both_correct": 1793, + "selected_both_wrong": 368 + }, + { + "budget": 0.48, + "threshold": -14.219049453735352, + "operator": "ge", + "actual_fraction": 0.5176044183638246, + "metric": 0.7533655505695547, + "rescued": 224, + "harmed": 577, + "missed_rescue": 141, + "avoided_harm": 1330, + "selected_both_correct": 1825, + "selected_both_wrong": 373 + }, + { + "budget": 0.49, + "threshold": -14.446796417236328, + "operator": "ge", + "actual_fraction": 0.5312392129789437, + "metric": 0.7512944425267518, + "rescued": 231, + "harmed": 596, + "missed_rescue": 134, + "avoided_harm": 1311, + "selected_both_correct": 1869, + "selected_both_wrong": 382 + }, + { + "budget": 0.5, + "threshold": -14.658285140991211, + "operator": "ge", + "actual_fraction": 0.5405591991715568, + "metric": 0.7474974111149465, + "rescued": 233, + "harmed": 620, + "missed_rescue": 132, + "avoided_harm": 1287, + "selected_both_correct": 1892, + "selected_both_wrong": 387 + }, + { + "budget": 0.51, + "threshold": -14.863032341003418, + "operator": "ge", + "actual_fraction": 0.5526406627545737, + "metric": 0.7437003797031412, + "rescued": 234, + "harmed": 643, + "missed_rescue": 131, + "avoided_harm": 1264, + "selected_both_correct": 1922, + "selected_both_wrong": 403 + }, + { + "budget": 0.52, + "threshold": -15.09691333770752, + "operator": "ge", + "actual_fraction": 0.5667932343803935, + "metric": 0.7385226095961339, + "rescued": 236, + "harmed": 675, + "missed_rescue": 129, + "avoided_harm": 1232, + "selected_both_correct": 1962, + "selected_both_wrong": 411 + }, + { + "budget": 0.53, + "threshold": -15.207231521606445, + "operator": "ge", + "actual_fraction": 0.572143596824301, + "metric": 0.7364515015533311, + "rescued": 237, + "harmed": 688, + "missed_rescue": 128, + "avoided_harm": 1219, + "selected_both_correct": 1976, + "selected_both_wrong": 414 + }, + { + "budget": 0.54, + "threshold": -15.392038345336914, + "operator": "ge", + "actual_fraction": 0.5854332067656196, + "metric": 0.732827062478426, + "rescued": 241, + "harmed": 713, + "missed_rescue": 124, + "avoided_harm": 1194, + "selected_both_correct": 2015, + "selected_both_wrong": 423 + }, + { + "budget": 0.55, + "threshold": -15.607501983642578, + "operator": "ge", + "actual_fraction": 0.5966517086641353, + "metric": 0.7304107697618226, + "rescued": 244, + "harmed": 730, + "missed_rescue": 121, + "avoided_harm": 1177, + "selected_both_correct": 2054, + "selected_both_wrong": 429 + }, + { + "budget": 0.56, + "threshold": -15.75269889831543, + "operator": "ge", + "actual_fraction": 0.6044183638246462, + "metric": 0.7285122540559199, + "rescued": 247, + "harmed": 744, + "missed_rescue": 118, + "avoided_harm": 1163, + "selected_both_correct": 2078, + "selected_both_wrong": 433 + }, + { + "budget": 0.57, + "threshold": -15.911096572875977, + "operator": "ge", + "actual_fraction": 0.6147739040386607, + "metric": 0.7262685536762168, + "rescued": 251, + "harmed": 761, + "missed_rescue": 114, + "avoided_harm": 1146, + "selected_both_correct": 2109, + "selected_both_wrong": 441 + }, + { + "budget": 0.58, + "threshold": -16.042922973632812, + "operator": "ge", + "actual_fraction": 0.6234035208836728, + "metric": 0.7247152226441146, + "rescued": 255, + "harmed": 774, + "missed_rescue": 110, + "avoided_harm": 1133, + "selected_both_correct": 2134, + "selected_both_wrong": 449 + }, + { + "budget": 0.59, + "threshold": -16.200180053710938, + "operator": "ge", + "actual_fraction": 0.6323783224024854, + "metric": 0.7217811529168104, + "rescued": 259, + "harmed": 795, + "missed_rescue": 106, + "avoided_harm": 1112, + "selected_both_correct": 2153, + "selected_both_wrong": 457 + }, + { + "budget": 0.6, + "threshold": -16.35162925720215, + "operator": "ge", + "actual_fraction": 0.6413531239212978, + "metric": 0.7185018985157059, + "rescued": 260, + "harmed": 815, + "missed_rescue": 105, + "avoided_harm": 1092, + "selected_both_correct": 2178, + "selected_both_wrong": 463 + }, + { + "budget": 0.61, + "threshold": -16.515636444091797, + "operator": "ge", + "actual_fraction": 0.6505005177770107, + "metric": 0.7155678287884017, + "rescued": 264, + "harmed": 836, + "missed_rescue": 101, + "avoided_harm": 1071, + "selected_both_correct": 2199, + "selected_both_wrong": 470 + }, + { + "budget": 0.62, + "threshold": -16.80984878540039, + "operator": "ge", + "actual_fraction": 0.6637901277183293, + "metric": 0.7124611667241975, + "rescued": 268, + "harmed": 858, + "missed_rescue": 97, + "avoided_harm": 1049, + "selected_both_correct": 2244, + "selected_both_wrong": 476 + }, + { + "budget": 0.63, + "threshold": -17.124507904052734, + "operator": "ge", + "actual_fraction": 0.6794960303762513, + "metric": 0.7079737659647911, + "rescued": 274, + "harmed": 890, + "missed_rescue": 91, + "avoided_harm": 1017, + "selected_both_correct": 2290, + "selected_both_wrong": 483 + }, + { + "budget": 0.64, + "threshold": -17.39409637451172, + "operator": "ge", + "actual_fraction": 0.6933034173282706, + "metric": 0.7040041422160856, + "rescued": 280, + "harmed": 919, + "missed_rescue": 85, + "avoided_harm": 988, + "selected_both_correct": 2329, + "selected_both_wrong": 489 + }, + { + "budget": 0.65, + "threshold": -17.61226463317871, + "operator": "ge", + "actual_fraction": 0.7021056265101829, + "metric": 0.7021056265101829, + "rescued": 285, + "harmed": 935, + "missed_rescue": 80, + "avoided_harm": 972, + "selected_both_correct": 2355, + "selected_both_wrong": 493 + }, + { + "budget": 0.66, + "threshold": -17.802616119384766, + "operator": "ge", + "actual_fraction": 0.7131515360717984, + "metric": 0.6995167414566793, + "rescued": 289, + "harmed": 954, + "missed_rescue": 76, + "avoided_harm": 953, + "selected_both_correct": 2388, + "selected_both_wrong": 501 + }, + { + "budget": 0.67, + "threshold": -17.987302780151367, + "operator": "ge", + "actual_fraction": 0.7197100448740076, + "metric": 0.6971004487400759, + "rescued": 292, + "harmed": 971, + "missed_rescue": 73, + "avoided_harm": 936, + "selected_both_correct": 2402, + "selected_both_wrong": 505 + }, + { + "budget": 0.68, + "threshold": -18.25859832763672, + "operator": "ge", + "actual_fraction": 0.7329996548153263, + "metric": 0.6939937866758716, + "rescued": 301, + "harmed": 998, + "missed_rescue": 64, + "avoided_harm": 909, + "selected_both_correct": 2427, + "selected_both_wrong": 521 + }, + { + "budget": 0.69, + "threshold": -18.391502380371094, + "operator": "ge", + "actual_fraction": 0.7402485329651364, + "metric": 0.6912323092854677, + "rescued": 302, + "harmed": 1015, + "missed_rescue": 63, + "avoided_harm": 892, + "selected_both_correct": 2445, + "selected_both_wrong": 527 + }, + { + "budget": 0.7, + "threshold": -18.60244369506836, + "operator": "ge", + "actual_fraction": 0.7535381429064549, + "metric": 0.6862271315153607, + "rescued": 306, + "harmed": 1048, + "missed_rescue": 59, + "avoided_harm": 859, + "selected_both_correct": 2471, + "selected_both_wrong": 541 + }, + { + "budget": 0.71, + "threshold": -18.753427505493164, + "operator": "ge", + "actual_fraction": 0.7602692440455644, + "metric": 0.684328615809458, + "rescued": 311, + "harmed": 1064, + "missed_rescue": 54, + "avoided_harm": 843, + "selected_both_correct": 2487, + "selected_both_wrong": 543 + }, + { + "budget": 0.72, + "threshold": -18.970993041992188, + "operator": "ge", + "actual_fraction": 0.7687262685536762, + "metric": 0.679323438039351, + "rescued": 313, + "harmed": 1095, + "missed_rescue": 52, + "avoided_harm": 812, + "selected_both_correct": 2498, + "selected_both_wrong": 548 + }, + { + "budget": 0.73, + "threshold": -19.116729736328125, + "operator": "ge", + "actual_fraction": 0.7764929237141871, + "metric": 0.6777701070072488, + "rescued": 318, + "harmed": 1109, + "missed_rescue": 47, + "avoided_harm": 798, + "selected_both_correct": 2517, + "selected_both_wrong": 555 + }, + { + "budget": 0.74, + "threshold": -19.298297882080078, + "operator": "ge", + "actual_fraction": 0.7820158784949949, + "metric": 0.6751812219537453, + "rescued": 318, + "harmed": 1124, + "missed_rescue": 47, + "avoided_harm": 783, + "selected_both_correct": 2529, + "selected_both_wrong": 560 + }, + { + "budget": 0.75, + "threshold": -19.566133499145508, + "operator": "ge", + "actual_fraction": 0.7944425267518123, + "metric": 0.6708664135312392, + "rescued": 326, + "harmed": 1157, + "missed_rescue": 39, + "avoided_harm": 750, + "selected_both_correct": 2553, + "selected_both_wrong": 567 + }, + { + "budget": 0.76, + "threshold": -19.99453353881836, + "operator": "ge", + "actual_fraction": 0.809285467725233, + "metric": 0.66430790472903, + "rescued": 329, + "harmed": 1198, + "missed_rescue": 36, + "avoided_harm": 709, + "selected_both_correct": 2581, + "selected_both_wrong": 581 + }, + { + "budget": 0.77, + "threshold": -20.128440856933594, + "operator": "ge", + "actual_fraction": 0.8132550914739386, + "metric": 0.6618916120124266, + "rescued": 329, + "harmed": 1212, + "missed_rescue": 36, + "avoided_harm": 695, + "selected_both_correct": 2587, + "selected_both_wrong": 584 + }, + { + "budget": 0.78, + "threshold": -20.354900360107422, + "operator": "ge", + "actual_fraction": 0.8218847083189507, + "metric": 0.6580945806006213, + "rescued": 329, + "harmed": 1234, + "missed_rescue": 36, + "avoided_harm": 673, + "selected_both_correct": 2608, + "selected_both_wrong": 591 + }, + { + "budget": 0.79, + "threshold": -20.70088005065918, + "operator": "ge", + "actual_fraction": 0.8331032102174664, + "metric": 0.6522264411460131, + "rescued": 330, + "harmed": 1269, + "missed_rescue": 35, + "avoided_harm": 638, + "selected_both_correct": 2625, + "selected_both_wrong": 603 + }, + { + "budget": 0.8, + "threshold": -21.008874893188477, + "operator": "ge", + "actual_fraction": 0.8415602347255782, + "metric": 0.6482568173973076, + "rescued": 330, + "harmed": 1292, + "missed_rescue": 35, + "avoided_harm": 615, + "selected_both_correct": 2644, + "selected_both_wrong": 610 + }, + { + "budget": 0.81, + "threshold": -21.276046752929688, + "operator": "ge", + "actual_fraction": 0.8491542975491888, + "metric": 0.6448049706593028, + "rescued": 332, + "harmed": 1314, + "missed_rescue": 33, + "avoided_harm": 593, + "selected_both_correct": 2658, + "selected_both_wrong": 616 + }, + { + "budget": 0.82, + "threshold": -21.52347183227539, + "operator": "ge", + "actual_fraction": 0.8577839143942009, + "metric": 0.6401449775629962, + "rescued": 335, + "harmed": 1344, + "missed_rescue": 30, + "avoided_harm": 563, + "selected_both_correct": 2672, + "selected_both_wrong": 619 + }, + { + "budget": 0.83, + "threshold": -21.878074645996094, + "operator": "ge", + "actual_fraction": 0.8676216775975146, + "metric": 0.6351397997928891, + "rescued": 338, + "harmed": 1376, + "missed_rescue": 27, + "avoided_harm": 531, + "selected_both_correct": 2684, + "selected_both_wrong": 629 + }, + { + "budget": 0.84, + "threshold": -22.360363006591797, + "operator": "ge", + "actual_fraction": 0.8779772178115292, + "metric": 0.6304798066965827, + "rescued": 341, + "harmed": 1406, + "missed_rescue": 24, + "avoided_harm": 501, + "selected_both_correct": 2705, + "selected_both_wrong": 635 + }, + { + "budget": 0.85, + "threshold": -22.77996826171875, + "operator": "ge", + "actual_fraction": 0.8879875733517432, + "metric": 0.6251294442526751, + "rescued": 345, + "harmed": 1441, + "missed_rescue": 20, + "avoided_harm": 466, + "selected_both_correct": 2718, + "selected_both_wrong": 641 + }, + { + "budget": 0.86, + "threshold": -23.05141830444336, + "operator": "ge", + "actual_fraction": 0.8929927511218502, + "metric": 0.6216775975146703, + "rescued": 345, + "harmed": 1461, + "missed_rescue": 20, + "avoided_harm": 446, + "selected_both_correct": 2724, + "selected_both_wrong": 644 + }, + { + "budget": 0.87, + "threshold": -23.436256408691406, + "operator": "ge", + "actual_fraction": 0.9016223679668622, + "metric": 0.6158094580600622, + "rescued": 346, + "harmed": 1496, + "missed_rescue": 19, + "avoided_harm": 411, + "selected_both_correct": 2733, + "selected_both_wrong": 649 + }, + { + "budget": 0.88, + "threshold": -23.772342681884766, + "operator": "ge", + "actual_fraction": 0.9102519848118743, + "metric": 0.6101139109423541, + "rescued": 348, + "harmed": 1531, + "missed_rescue": 17, + "avoided_harm": 376, + "selected_both_correct": 2742, + "selected_both_wrong": 653 + }, + { + "budget": 0.89, + "threshold": -24.192012786865234, + "operator": "ge", + "actual_fraction": 0.9200897480151882, + "metric": 0.6054539178460476, + "rescued": 350, + "harmed": 1560, + "missed_rescue": 15, + "avoided_harm": 347, + "selected_both_correct": 2762, + "selected_both_wrong": 659 + }, + { + "budget": 0.9, + "threshold": -24.707534790039062, + "operator": "ge", + "actual_fraction": 0.9288919571971005, + "metric": 0.6004487400759406, + "rescued": 353, + "harmed": 1592, + "missed_rescue": 12, + "avoided_harm": 315, + "selected_both_correct": 2771, + "selected_both_wrong": 666 + }, + { + "budget": 0.91, + "threshold": -25.141036987304688, + "operator": "ge", + "actual_fraction": 0.9345875043148084, + "metric": 0.5971694856748361, + "rescued": 355, + "harmed": 1613, + "missed_rescue": 10, + "avoided_harm": 294, + "selected_both_correct": 2777, + "selected_both_wrong": 670 + }, + { + "budget": 0.92, + "threshold": -25.627573013305664, + "operator": "ge", + "actual_fraction": 0.9420089748015188, + "metric": 0.5923369002416292, + "rescued": 357, + "harmed": 1643, + "missed_rescue": 8, + "avoided_harm": 264, + "selected_both_correct": 2782, + "selected_both_wrong": 676 + }, + { + "budget": 0.93, + "threshold": -26.255355834960938, + "operator": "ge", + "actual_fraction": 0.9497756299620297, + "metric": 0.5873317224715222, + "rescued": 357, + "harmed": 1672, + "missed_rescue": 8, + "avoided_harm": 235, + "selected_both_correct": 2793, + "selected_both_wrong": 681 + }, + { + "budget": 0.94, + "threshold": -26.890518188476562, + "operator": "ge", + "actual_fraction": 0.9559889540904384, + "metric": 0.5831895063859165, + "rescued": 359, + "harmed": 1698, + "missed_rescue": 6, + "avoided_harm": 209, + "selected_both_correct": 2797, + "selected_both_wrong": 685 + }, + { + "budget": 0.95, + "threshold": -27.40430450439453, + "operator": "ge", + "actual_fraction": 0.9613393165343459, + "metric": 0.5788746979634104, + "rescued": 360, + "harmed": 1724, + "missed_rescue": 5, + "avoided_harm": 183, + "selected_both_correct": 2798, + "selected_both_wrong": 688 + }, + { + "budget": 0.96, + "threshold": -27.9757022857666, + "operator": "ge", + "actual_fraction": 0.9666896789782534, + "metric": 0.5750776665516051, + "rescued": 360, + "harmed": 1746, + "missed_rescue": 5, + "avoided_harm": 161, + "selected_both_correct": 2802, + "selected_both_wrong": 693 + }, + { + "budget": 0.97, + "threshold": -29.13153839111328, + "operator": "ge", + "actual_fraction": 0.9767000345184674, + "metric": 0.567828788401795, + "rescued": 362, + "harmed": 1790, + "missed_rescue": 3, + "avoided_harm": 117, + "selected_both_correct": 2806, + "selected_both_wrong": 701 + }, + { + "budget": 0.98, + "threshold": -30.387767791748047, + "operator": "ge", + "actual_fraction": 0.9842940973420781, + "metric": 0.5610976872626855, + "rescued": 363, + "harmed": 1830, + "missed_rescue": 2, + "avoided_harm": 77, + "selected_both_correct": 2807, + "selected_both_wrong": 703 + }, + { + "budget": 0.99, + "threshold": -32.89930725097656, + "operator": "ge", + "actual_fraction": 0.9946496375560925, + "metric": 0.5526406627545737, + "rescued": 364, + "harmed": 1880, + "missed_rescue": 1, + "avoided_harm": 27, + "selected_both_correct": 2810, + "selected_both_wrong": 709 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5481532619951675, + "rescued": 365, + "harmed": 1907, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 2811, + "selected_both_wrong": 711 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 659, + "rescuable": 2, + "harmful": 524, + "both_wrong": 15, + "unit": "sample", + "total": 1200 + }, + "test": { + "both_correct": 2811, + "rescuable": 365, + "harmful": 1907, + "both_wrong": 711, + "unit": "sample", + "total": 5794 + } + }, + "small": { + "validation": 0.9858333333333333, + "test": 0.81429064549534 + }, + "large": { + "validation": 0.5508333333333333, + "test": 0.5481532619951675 + }, + "learned_minus_confidence_test": -0.0024162927166033743, + "learned_minus_small_test": -0.0008629616845011734, + "learned_minus_large_test": 0.2652744218156714 + }, + { + "key": [ + "cub", + "glsim", + "MiniCPM" + ], + "expert_display": "GLSim", + "run": "shared_12_glsim_MiniCPM", + "validation_rows": 1200, + "test_rows": 5794, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/glsim/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 1, + "validation_area_0_25": 0.9005, + "validation_point": { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.985, + "rescued": 1, + "harmed": 6, + "missed_rescue": 2, + "avoided_harm": 851, + "selected_both_correct": 0, + "selected_both_wrong": 5 + }, + "test_point": { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.08698653779772178, + "metric": 0.8769416637901277, + "rescued": 50, + "harmed": 193, + "missed_rescue": 56, + "avoided_harm": 3598, + "selected_both_correct": 42, + "selected_both_wrong": 219 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9891666666666666, + "rescued": 0, + "harmed": 0, + "missed_rescue": 3, + "avoided_harm": 857, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.985, + "rescued": 1, + "harmed": 6, + "missed_rescue": 2, + "avoided_harm": 851, + "selected_both_correct": 0, + "selected_both_wrong": 5 + }, + { + "budget": 0.02, + "threshold": -0.6014261841773987, + "operator": "ge", + "actual_fraction": 0.02, + "metric": 0.9791666666666666, + "rescued": 2, + "harmed": 14, + "missed_rescue": 1, + "avoided_harm": 843, + "selected_both_correct": 2, + "selected_both_wrong": 6 + }, + { + "budget": 0.03, + "threshold": -0.6401472687721252, + "operator": "ge", + "actual_fraction": 0.03, + "metric": 0.9716666666666667, + "rescued": 2, + "harmed": 23, + "missed_rescue": 1, + "avoided_harm": 834, + "selected_both_correct": 3, + "selected_both_wrong": 8 + }, + { + "budget": 0.04, + "threshold": -0.6884650588035583, + "operator": "ge", + "actual_fraction": 0.04, + "metric": 0.9658333333333333, + "rescued": 3, + "harmed": 31, + "missed_rescue": 0, + "avoided_harm": 826, + "selected_both_correct": 5, + "selected_both_wrong": 9 + }, + { + "budget": 0.05, + "threshold": -0.7208121418952942, + "operator": "ge", + "actual_fraction": 0.05, + "metric": 0.9591666666666666, + "rescued": 3, + "harmed": 39, + "missed_rescue": 0, + "avoided_harm": 818, + "selected_both_correct": 9, + "selected_both_wrong": 9 + }, + { + "budget": 0.06, + "threshold": -0.7578181624412537, + "operator": "ge", + "actual_fraction": 0.06, + "metric": 0.9516666666666667, + "rescued": 3, + "harmed": 48, + "missed_rescue": 0, + "avoided_harm": 809, + "selected_both_correct": 12, + "selected_both_wrong": 9 + }, + { + "budget": 0.07, + "threshold": -0.7790470719337463, + "operator": "ge", + "actual_fraction": 0.07, + "metric": 0.9441666666666667, + "rescued": 3, + "harmed": 57, + "missed_rescue": 0, + "avoided_harm": 800, + "selected_both_correct": 15, + "selected_both_wrong": 9 + }, + { + "budget": 0.08, + "threshold": -0.7927228808403015, + "operator": "ge", + "actual_fraction": 0.08, + "metric": 0.9375, + "rescued": 3, + "harmed": 65, + "missed_rescue": 0, + "avoided_harm": 792, + "selected_both_correct": 18, + "selected_both_wrong": 10 + }, + { + "budget": 0.09, + "threshold": -0.8049252033233643, + "operator": "ge", + "actual_fraction": 0.09, + "metric": 0.93, + "rescued": 3, + "harmed": 74, + "missed_rescue": 0, + "avoided_harm": 783, + "selected_both_correct": 21, + "selected_both_wrong": 10 + }, + { + "budget": 0.1, + "threshold": -0.80850750207901, + "operator": "ge", + "actual_fraction": 0.1, + "metric": 0.9216666666666666, + "rescued": 3, + "harmed": 84, + "missed_rescue": 0, + "avoided_harm": 773, + "selected_both_correct": 23, + "selected_both_wrong": 10 + }, + { + "budget": 0.11, + "threshold": -0.819871723651886, + "operator": "ge", + "actual_fraction": 0.11, + "metric": 0.9133333333333333, + "rescued": 3, + "harmed": 94, + "missed_rescue": 0, + "avoided_harm": 763, + "selected_both_correct": 25, + "selected_both_wrong": 10 + }, + { + "budget": 0.12, + "threshold": -0.8310170769691467, + "operator": "ge", + "actual_fraction": 0.12, + "metric": 0.905, + "rescued": 3, + "harmed": 104, + "missed_rescue": 0, + "avoided_harm": 753, + "selected_both_correct": 27, + "selected_both_wrong": 10 + }, + { + "budget": 0.13, + "threshold": -0.8376906514167786, + "operator": "ge", + "actual_fraction": 0.13, + "metric": 0.8966666666666666, + "rescued": 3, + "harmed": 114, + "missed_rescue": 0, + "avoided_harm": 743, + "selected_both_correct": 29, + "selected_both_wrong": 10 + }, + { + "budget": 0.14, + "threshold": -0.8419997692108154, + "operator": "ge", + "actual_fraction": 0.14, + "metric": 0.8875, + "rescued": 3, + "harmed": 125, + "missed_rescue": 0, + "avoided_harm": 732, + "selected_both_correct": 30, + "selected_both_wrong": 10 + }, + { + "budget": 0.15, + "threshold": -0.8447275161743164, + "operator": "ge", + "actual_fraction": 0.15, + "metric": 0.8808333333333334, + "rescued": 3, + "harmed": 133, + "missed_rescue": 0, + "avoided_harm": 724, + "selected_both_correct": 34, + "selected_both_wrong": 10 + }, + { + "budget": 0.16, + "threshold": -0.8506776690483093, + "operator": "ge", + "actual_fraction": 0.16, + "metric": 0.8716666666666667, + "rescued": 3, + "harmed": 144, + "missed_rescue": 0, + "avoided_harm": 713, + "selected_both_correct": 35, + "selected_both_wrong": 10 + }, + { + "budget": 0.17, + "threshold": -0.8545639514923096, + "operator": "ge", + "actual_fraction": 0.17, + "metric": 0.8633333333333333, + "rescued": 3, + "harmed": 154, + "missed_rescue": 0, + "avoided_harm": 703, + "selected_both_correct": 37, + "selected_both_wrong": 10 + }, + { + "budget": 0.18, + "threshold": -0.8578683137893677, + "operator": "ge", + "actual_fraction": 0.18, + "metric": 0.855, + "rescued": 3, + "harmed": 164, + "missed_rescue": 0, + "avoided_harm": 693, + "selected_both_correct": 39, + "selected_both_wrong": 10 + }, + { + "budget": 0.19, + "threshold": -0.86090087890625, + "operator": "ge", + "actual_fraction": 0.19, + "metric": 0.8491666666666666, + "rescued": 3, + "harmed": 171, + "missed_rescue": 0, + "avoided_harm": 686, + "selected_both_correct": 44, + "selected_both_wrong": 10 + }, + { + "budget": 0.2, + "threshold": -0.8640955090522766, + "operator": "ge", + "actual_fraction": 0.2, + "metric": 0.8425, + "rescued": 3, + "harmed": 179, + "missed_rescue": 0, + "avoided_harm": 678, + "selected_both_correct": 48, + "selected_both_wrong": 10 + }, + { + "budget": 0.21, + "threshold": -0.8664237260818481, + "operator": "ge", + "actual_fraction": 0.21, + "metric": 0.8358333333333333, + "rescued": 3, + "harmed": 187, + "missed_rescue": 0, + "avoided_harm": 670, + "selected_both_correct": 52, + "selected_both_wrong": 10 + }, + { + "budget": 0.22, + "threshold": -0.8686786890029907, + "operator": "ge", + "actual_fraction": 0.22, + "metric": 0.8283333333333334, + "rescued": 3, + "harmed": 196, + "missed_rescue": 0, + "avoided_harm": 661, + "selected_both_correct": 55, + "selected_both_wrong": 10 + }, + { + "budget": 0.23, + "threshold": -0.8709638118743896, + "operator": "ge", + "actual_fraction": 0.23, + "metric": 0.8208333333333333, + "rescued": 3, + "harmed": 205, + "missed_rescue": 0, + "avoided_harm": 652, + "selected_both_correct": 58, + "selected_both_wrong": 10 + }, + { + "budget": 0.24, + "threshold": -0.8724074959754944, + "operator": "ge", + "actual_fraction": 0.24, + "metric": 0.8158333333333333, + "rescued": 3, + "harmed": 211, + "missed_rescue": 0, + "avoided_harm": 646, + "selected_both_correct": 64, + "selected_both_wrong": 10 + }, + { + "budget": 0.25, + "threshold": -0.8736594319343567, + "operator": "ge", + "actual_fraction": 0.25, + "metric": 0.8075, + "rescued": 3, + "harmed": 221, + "missed_rescue": 0, + "avoided_harm": 636, + "selected_both_correct": 66, + "selected_both_wrong": 10 + }, + { + "budget": 0.26, + "threshold": -0.8752554059028625, + "operator": "ge", + "actual_fraction": 0.26, + "metric": 0.8016666666666666, + "rescued": 3, + "harmed": 228, + "missed_rescue": 0, + "avoided_harm": 629, + "selected_both_correct": 71, + "selected_both_wrong": 10 + }, + { + "budget": 0.27, + "threshold": -0.8766024112701416, + "operator": "ge", + "actual_fraction": 0.27, + "metric": 0.7933333333333333, + "rescued": 3, + "harmed": 238, + "missed_rescue": 0, + "avoided_harm": 619, + "selected_both_correct": 73, + "selected_both_wrong": 10 + }, + { + "budget": 0.28, + "threshold": -0.878311276435852, + "operator": "ge", + "actual_fraction": 0.28, + "metric": 0.7866666666666666, + "rescued": 3, + "harmed": 246, + "missed_rescue": 0, + "avoided_harm": 611, + "selected_both_correct": 77, + "selected_both_wrong": 10 + }, + { + "budget": 0.29, + "threshold": -0.880103588104248, + "operator": "ge", + "actual_fraction": 0.29, + "metric": 0.78, + "rescued": 3, + "harmed": 254, + "missed_rescue": 0, + "avoided_harm": 603, + "selected_both_correct": 81, + "selected_both_wrong": 10 + }, + { + "budget": 0.3, + "threshold": -0.8824952244758606, + "operator": "ge", + "actual_fraction": 0.3, + "metric": 0.7725, + "rescued": 3, + "harmed": 263, + "missed_rescue": 0, + "avoided_harm": 594, + "selected_both_correct": 84, + "selected_both_wrong": 10 + }, + { + "budget": 0.31, + "threshold": -0.8831692337989807, + "operator": "ge", + "actual_fraction": 0.31, + "metric": 0.7658333333333334, + "rescued": 3, + "harmed": 271, + "missed_rescue": 0, + "avoided_harm": 586, + "selected_both_correct": 88, + "selected_both_wrong": 10 + }, + { + "budget": 0.32, + "threshold": -0.8845130205154419, + "operator": "ge", + "actual_fraction": 0.32, + "metric": 0.76, + "rescued": 3, + "harmed": 278, + "missed_rescue": 0, + "avoided_harm": 579, + "selected_both_correct": 93, + "selected_both_wrong": 10 + }, + { + "budget": 0.33, + "threshold": -0.8855781555175781, + "operator": "ge", + "actual_fraction": 0.33, + "metric": 0.7508333333333334, + "rescued": 3, + "harmed": 289, + "missed_rescue": 0, + "avoided_harm": 568, + "selected_both_correct": 94, + "selected_both_wrong": 10 + }, + { + "budget": 0.34, + "threshold": -0.8865919709205627, + "operator": "ge", + "actual_fraction": 0.34, + "metric": 0.7433333333333333, + "rescued": 3, + "harmed": 298, + "missed_rescue": 0, + "avoided_harm": 559, + "selected_both_correct": 97, + "selected_both_wrong": 10 + }, + { + "budget": 0.35, + "threshold": -0.8877584338188171, + "operator": "ge", + "actual_fraction": 0.35, + "metric": 0.735, + "rescued": 3, + "harmed": 308, + "missed_rescue": 0, + "avoided_harm": 549, + "selected_both_correct": 99, + "selected_both_wrong": 10 + }, + { + "budget": 0.36, + "threshold": -0.8890330195426941, + "operator": "ge", + "actual_fraction": 0.36, + "metric": 0.7291666666666666, + "rescued": 3, + "harmed": 315, + "missed_rescue": 0, + "avoided_harm": 542, + "selected_both_correct": 104, + "selected_both_wrong": 10 + }, + { + "budget": 0.37, + "threshold": -0.890850305557251, + "operator": "ge", + "actual_fraction": 0.37, + "metric": 0.7225, + "rescued": 3, + "harmed": 323, + "missed_rescue": 0, + "avoided_harm": 534, + "selected_both_correct": 108, + "selected_both_wrong": 10 + }, + { + "budget": 0.38, + "threshold": -0.8916605710983276, + "operator": "ge", + "actual_fraction": 0.38, + "metric": 0.7183333333333334, + "rescued": 3, + "harmed": 328, + "missed_rescue": 0, + "avoided_harm": 529, + "selected_both_correct": 115, + "selected_both_wrong": 10 + }, + { + "budget": 0.39, + "threshold": -0.8928589224815369, + "operator": "ge", + "actual_fraction": 0.39, + "metric": 0.7116666666666667, + "rescued": 3, + "harmed": 336, + "missed_rescue": 0, + "avoided_harm": 521, + "selected_both_correct": 119, + "selected_both_wrong": 10 + }, + { + "budget": 0.4, + "threshold": -0.8940943479537964, + "operator": "ge", + "actual_fraction": 0.4, + "metric": 0.7033333333333334, + "rescued": 3, + "harmed": 346, + "missed_rescue": 0, + "avoided_harm": 511, + "selected_both_correct": 121, + "selected_both_wrong": 10 + }, + { + "budget": 0.41, + "threshold": -0.8949463963508606, + "operator": "ge", + "actual_fraction": 0.4091666666666667, + "metric": 0.6983333333333334, + "rescued": 3, + "harmed": 352, + "missed_rescue": 0, + "avoided_harm": 505, + "selected_both_correct": 126, + "selected_both_wrong": 10 + }, + { + "budget": 0.42, + "threshold": -0.8960229158401489, + "operator": "ge", + "actual_fraction": 0.42, + "metric": 0.6916666666666667, + "rescued": 3, + "harmed": 360, + "missed_rescue": 0, + "avoided_harm": 497, + "selected_both_correct": 131, + "selected_both_wrong": 10 + }, + { + "budget": 0.43, + "threshold": -0.8966536521911621, + "operator": "ge", + "actual_fraction": 0.43, + "metric": 0.6841666666666667, + "rescued": 3, + "harmed": 369, + "missed_rescue": 0, + "avoided_harm": 488, + "selected_both_correct": 134, + "selected_both_wrong": 10 + }, + { + "budget": 0.44, + "threshold": -0.8975363373756409, + "operator": "ge", + "actual_fraction": 0.44, + "metric": 0.68, + "rescued": 3, + "harmed": 374, + "missed_rescue": 0, + "avoided_harm": 483, + "selected_both_correct": 141, + "selected_both_wrong": 10 + }, + { + "budget": 0.45, + "threshold": -0.8988122940063477, + "operator": "ge", + "actual_fraction": 0.45, + "metric": 0.6741666666666667, + "rescued": 3, + "harmed": 381, + "missed_rescue": 0, + "avoided_harm": 476, + "selected_both_correct": 146, + "selected_both_wrong": 10 + }, + { + "budget": 0.46, + "threshold": -0.9000959396362305, + "operator": "ge", + "actual_fraction": 0.46, + "metric": 0.665, + "rescued": 3, + "harmed": 392, + "missed_rescue": 0, + "avoided_harm": 465, + "selected_both_correct": 147, + "selected_both_wrong": 10 + }, + { + "budget": 0.47, + "threshold": -0.9013051986694336, + "operator": "ge", + "actual_fraction": 0.47, + "metric": 0.6583333333333333, + "rescued": 3, + "harmed": 400, + "missed_rescue": 0, + "avoided_harm": 457, + "selected_both_correct": 151, + "selected_both_wrong": 10 + }, + { + "budget": 0.48, + "threshold": -0.9029324054718018, + "operator": "ge", + "actual_fraction": 0.48, + "metric": 0.6525, + "rescued": 3, + "harmed": 407, + "missed_rescue": 0, + "avoided_harm": 450, + "selected_both_correct": 156, + "selected_both_wrong": 10 + }, + { + "budget": 0.49, + "threshold": -0.9036871790885925, + "operator": "ge", + "actual_fraction": 0.49, + "metric": 0.6475, + "rescued": 3, + "harmed": 413, + "missed_rescue": 0, + "avoided_harm": 444, + "selected_both_correct": 162, + "selected_both_wrong": 10 + }, + { + "budget": 0.5, + "threshold": -0.9044621586799622, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.6425, + "rescued": 3, + "harmed": 419, + "missed_rescue": 0, + "avoided_harm": 438, + "selected_both_correct": 168, + "selected_both_wrong": 10 + }, + { + "budget": 0.51, + "threshold": -0.9054575562477112, + "operator": "ge", + "actual_fraction": 0.51, + "metric": 0.6358333333333334, + "rescued": 3, + "harmed": 427, + "missed_rescue": 0, + "avoided_harm": 430, + "selected_both_correct": 172, + "selected_both_wrong": 10 + }, + { + "budget": 0.52, + "threshold": -0.906682550907135, + "operator": "ge", + "actual_fraction": 0.52, + "metric": 0.6266666666666667, + "rescued": 3, + "harmed": 438, + "missed_rescue": 0, + "avoided_harm": 419, + "selected_both_correct": 173, + "selected_both_wrong": 10 + }, + { + "budget": 0.53, + "threshold": -0.9072114825248718, + "operator": "ge", + "actual_fraction": 0.53, + "metric": 0.6225, + "rescued": 3, + "harmed": 443, + "missed_rescue": 0, + "avoided_harm": 414, + "selected_both_correct": 180, + "selected_both_wrong": 10 + }, + { + "budget": 0.54, + "threshold": -0.9083563089370728, + "operator": "ge", + "actual_fraction": 0.54, + "metric": 0.6158333333333333, + "rescued": 3, + "harmed": 451, + "missed_rescue": 0, + "avoided_harm": 406, + "selected_both_correct": 184, + "selected_both_wrong": 10 + }, + { + "budget": 0.55, + "threshold": -0.9095192551612854, + "operator": "ge", + "actual_fraction": 0.55, + "metric": 0.6091666666666666, + "rescued": 3, + "harmed": 459, + "missed_rescue": 0, + "avoided_harm": 398, + "selected_both_correct": 188, + "selected_both_wrong": 10 + }, + { + "budget": 0.56, + "threshold": -0.9106979370117188, + "operator": "ge", + "actual_fraction": 0.56, + "metric": 0.6016666666666667, + "rescued": 3, + "harmed": 468, + "missed_rescue": 0, + "avoided_harm": 389, + "selected_both_correct": 191, + "selected_both_wrong": 10 + }, + { + "budget": 0.57, + "threshold": -0.9115362167358398, + "operator": "ge", + "actual_fraction": 0.5691666666666667, + "metric": 0.595, + "rescued": 3, + "harmed": 476, + "missed_rescue": 0, + "avoided_harm": 381, + "selected_both_correct": 194, + "selected_both_wrong": 10 + }, + { + "budget": 0.58, + "threshold": -0.9124338030815125, + "operator": "ge", + "actual_fraction": 0.58, + "metric": 0.5866666666666667, + "rescued": 3, + "harmed": 486, + "missed_rescue": 0, + "avoided_harm": 371, + "selected_both_correct": 197, + "selected_both_wrong": 10 + }, + { + "budget": 0.59, + "threshold": -0.9134801626205444, + "operator": "ge", + "actual_fraction": 0.59, + "metric": 0.5783333333333334, + "rescued": 3, + "harmed": 496, + "missed_rescue": 0, + "avoided_harm": 361, + "selected_both_correct": 199, + "selected_both_wrong": 10 + }, + { + "budget": 0.6, + "threshold": -0.9143747091293335, + "operator": "ge", + "actual_fraction": 0.6, + "metric": 0.5733333333333334, + "rescued": 3, + "harmed": 502, + "missed_rescue": 0, + "avoided_harm": 355, + "selected_both_correct": 205, + "selected_both_wrong": 10 + }, + { + "budget": 0.61, + "threshold": -0.915804386138916, + "operator": "ge", + "actual_fraction": 0.61, + "metric": 0.5641666666666667, + "rescued": 3, + "harmed": 513, + "missed_rescue": 0, + "avoided_harm": 344, + "selected_both_correct": 206, + "selected_both_wrong": 10 + }, + { + "budget": 0.62, + "threshold": -0.9165146350860596, + "operator": "ge", + "actual_fraction": 0.62, + "metric": 0.555, + "rescued": 3, + "harmed": 524, + "missed_rescue": 0, + "avoided_harm": 333, + "selected_both_correct": 207, + "selected_both_wrong": 10 + }, + { + "budget": 0.63, + "threshold": -0.9172822833061218, + "operator": "ge", + "actual_fraction": 0.63, + "metric": 0.55, + "rescued": 3, + "harmed": 530, + "missed_rescue": 0, + "avoided_harm": 327, + "selected_both_correct": 213, + "selected_both_wrong": 10 + }, + { + "budget": 0.64, + "threshold": -0.9181071519851685, + "operator": "ge", + "actual_fraction": 0.64, + "metric": 0.5416666666666666, + "rescued": 3, + "harmed": 540, + "missed_rescue": 0, + "avoided_harm": 317, + "selected_both_correct": 215, + "selected_both_wrong": 10 + }, + { + "budget": 0.65, + "threshold": -0.9192029237747192, + "operator": "ge", + "actual_fraction": 0.65, + "metric": 0.5383333333333333, + "rescued": 3, + "harmed": 544, + "missed_rescue": 0, + "avoided_harm": 313, + "selected_both_correct": 223, + "selected_both_wrong": 10 + }, + { + "budget": 0.66, + "threshold": -0.9200543761253357, + "operator": "ge", + "actual_fraction": 0.66, + "metric": 0.5333333333333333, + "rescued": 3, + "harmed": 550, + "missed_rescue": 0, + "avoided_harm": 307, + "selected_both_correct": 229, + "selected_both_wrong": 10 + }, + { + "budget": 0.67, + "threshold": -0.9208071231842041, + "operator": "ge", + "actual_fraction": 0.67, + "metric": 0.5266666666666666, + "rescued": 3, + "harmed": 558, + "missed_rescue": 0, + "avoided_harm": 299, + "selected_both_correct": 233, + "selected_both_wrong": 10 + }, + { + "budget": 0.68, + "threshold": -0.9215553998947144, + "operator": "ge", + "actual_fraction": 0.68, + "metric": 0.5175, + "rescued": 3, + "harmed": 569, + "missed_rescue": 0, + "avoided_harm": 288, + "selected_both_correct": 234, + "selected_both_wrong": 10 + }, + { + "budget": 0.69, + "threshold": -0.9220835566520691, + "operator": "ge", + "actual_fraction": 0.6891666666666667, + "metric": 0.5116666666666667, + "rescued": 3, + "harmed": 576, + "missed_rescue": 0, + "avoided_harm": 281, + "selected_both_correct": 238, + "selected_both_wrong": 10 + }, + { + "budget": 0.7, + "threshold": -0.9232237935066223, + "operator": "ge", + "actual_fraction": 0.7, + "metric": 0.5041666666666667, + "rescued": 3, + "harmed": 585, + "missed_rescue": 0, + "avoided_harm": 272, + "selected_both_correct": 242, + "selected_both_wrong": 10 + }, + { + "budget": 0.71, + "threshold": -0.9243364334106445, + "operator": "ge", + "actual_fraction": 0.71, + "metric": 0.49833333333333335, + "rescued": 3, + "harmed": 592, + "missed_rescue": 0, + "avoided_harm": 265, + "selected_both_correct": 247, + "selected_both_wrong": 10 + }, + { + "budget": 0.72, + "threshold": -0.9252541065216064, + "operator": "ge", + "actual_fraction": 0.72, + "metric": 0.49083333333333334, + "rescued": 3, + "harmed": 601, + "missed_rescue": 0, + "avoided_harm": 256, + "selected_both_correct": 250, + "selected_both_wrong": 10 + }, + { + "budget": 0.73, + "threshold": -0.925781786441803, + "operator": "ge", + "actual_fraction": 0.73, + "metric": 0.485, + "rescued": 3, + "harmed": 608, + "missed_rescue": 0, + "avoided_harm": 249, + "selected_both_correct": 255, + "selected_both_wrong": 10 + }, + { + "budget": 0.74, + "threshold": -0.9274963140487671, + "operator": "ge", + "actual_fraction": 0.74, + "metric": 0.47833333333333333, + "rescued": 3, + "harmed": 616, + "missed_rescue": 0, + "avoided_harm": 241, + "selected_both_correct": 259, + "selected_both_wrong": 10 + }, + { + "budget": 0.75, + "threshold": -0.9286423325538635, + "operator": "ge", + "actual_fraction": 0.75, + "metric": 0.4725, + "rescued": 3, + "harmed": 623, + "missed_rescue": 0, + "avoided_harm": 234, + "selected_both_correct": 264, + "selected_both_wrong": 10 + }, + { + "budget": 0.76, + "threshold": -0.9301291704177856, + "operator": "ge", + "actual_fraction": 0.76, + "metric": 0.4666666666666667, + "rescued": 3, + "harmed": 630, + "missed_rescue": 0, + "avoided_harm": 227, + "selected_both_correct": 269, + "selected_both_wrong": 10 + }, + { + "budget": 0.77, + "threshold": -0.9318777918815613, + "operator": "ge", + "actual_fraction": 0.77, + "metric": 0.4583333333333333, + "rescued": 3, + "harmed": 640, + "missed_rescue": 0, + "avoided_harm": 217, + "selected_both_correct": 271, + "selected_both_wrong": 10 + }, + { + "budget": 0.78, + "threshold": -0.9332055449485779, + "operator": "ge", + "actual_fraction": 0.78, + "metric": 0.45, + "rescued": 3, + "harmed": 650, + "missed_rescue": 0, + "avoided_harm": 207, + "selected_both_correct": 273, + "selected_both_wrong": 10 + }, + { + "budget": 0.79, + "threshold": -0.9344666600227356, + "operator": "ge", + "actual_fraction": 0.79, + "metric": 0.4425, + "rescued": 3, + "harmed": 659, + "missed_rescue": 0, + "avoided_harm": 198, + "selected_both_correct": 276, + "selected_both_wrong": 10 + }, + { + "budget": 0.8, + "threshold": -0.9362465143203735, + "operator": "ge", + "actual_fraction": 0.8, + "metric": 0.43583333333333335, + "rescued": 3, + "harmed": 667, + "missed_rescue": 0, + "avoided_harm": 190, + "selected_both_correct": 280, + "selected_both_wrong": 10 + }, + { + "budget": 0.81, + "threshold": -0.937842071056366, + "operator": "ge", + "actual_fraction": 0.81, + "metric": 0.4275, + "rescued": 3, + "harmed": 677, + "missed_rescue": 0, + "avoided_harm": 180, + "selected_both_correct": 282, + "selected_both_wrong": 10 + }, + { + "budget": 0.82, + "threshold": -0.9384457468986511, + "operator": "ge", + "actual_fraction": 0.8191666666666667, + "metric": 0.4216666666666667, + "rescued": 3, + "harmed": 684, + "missed_rescue": 0, + "avoided_harm": 173, + "selected_both_correct": 286, + "selected_both_wrong": 10 + }, + { + "budget": 0.83, + "threshold": -0.9396296143531799, + "operator": "ge", + "actual_fraction": 0.83, + "metric": 0.41333333333333333, + "rescued": 3, + "harmed": 694, + "missed_rescue": 0, + "avoided_harm": 163, + "selected_both_correct": 289, + "selected_both_wrong": 10 + }, + { + "budget": 0.84, + "threshold": -0.9407488703727722, + "operator": "ge", + "actual_fraction": 0.84, + "metric": 0.4058333333333333, + "rescued": 3, + "harmed": 703, + "missed_rescue": 0, + "avoided_harm": 154, + "selected_both_correct": 292, + "selected_both_wrong": 10 + }, + { + "budget": 0.85, + "threshold": -0.942738950252533, + "operator": "ge", + "actual_fraction": 0.85, + "metric": 0.4, + "rescued": 3, + "harmed": 710, + "missed_rescue": 0, + "avoided_harm": 147, + "selected_both_correct": 297, + "selected_both_wrong": 10 + }, + { + "budget": 0.86, + "threshold": -0.9438527226448059, + "operator": "gt", + "actual_fraction": 0.8591666666666666, + "metric": 0.39416666666666667, + "rescued": 3, + "harmed": 717, + "missed_rescue": 0, + "avoided_harm": 140, + "selected_both_correct": 301, + "selected_both_wrong": 10 + }, + { + "budget": 0.87, + "threshold": -0.9450756907463074, + "operator": "ge", + "actual_fraction": 0.87, + "metric": 0.38916666666666666, + "rescued": 3, + "harmed": 723, + "missed_rescue": 0, + "avoided_harm": 134, + "selected_both_correct": 308, + "selected_both_wrong": 10 + }, + { + "budget": 0.88, + "threshold": -0.9481598138809204, + "operator": "ge", + "actual_fraction": 0.88, + "metric": 0.38083333333333336, + "rescued": 3, + "harmed": 733, + "missed_rescue": 0, + "avoided_harm": 124, + "selected_both_correct": 310, + "selected_both_wrong": 10 + }, + { + "budget": 0.89, + "threshold": -0.9500836730003357, + "operator": "ge", + "actual_fraction": 0.89, + "metric": 0.37333333333333335, + "rescued": 3, + "harmed": 742, + "missed_rescue": 0, + "avoided_harm": 115, + "selected_both_correct": 313, + "selected_both_wrong": 10 + }, + { + "budget": 0.9, + "threshold": -0.9519118666648865, + "operator": "ge", + "actual_fraction": 0.9, + "metric": 0.365, + "rescued": 3, + "harmed": 752, + "missed_rescue": 0, + "avoided_harm": 105, + "selected_both_correct": 315, + "selected_both_wrong": 10 + }, + { + "budget": 0.91, + "threshold": -0.9543898701667786, + "operator": "ge", + "actual_fraction": 0.91, + "metric": 0.3566666666666667, + "rescued": 3, + "harmed": 762, + "missed_rescue": 0, + "avoided_harm": 95, + "selected_both_correct": 317, + "selected_both_wrong": 10 + }, + { + "budget": 0.92, + "threshold": -0.9570822715759277, + "operator": "ge", + "actual_fraction": 0.92, + "metric": 0.34833333333333333, + "rescued": 3, + "harmed": 772, + "missed_rescue": 0, + "avoided_harm": 85, + "selected_both_correct": 319, + "selected_both_wrong": 10 + }, + { + "budget": 0.93, + "threshold": -0.9593446254730225, + "operator": "ge", + "actual_fraction": 0.93, + "metric": 0.3383333333333333, + "rescued": 3, + "harmed": 784, + "missed_rescue": 0, + "avoided_harm": 73, + "selected_both_correct": 319, + "selected_both_wrong": 10 + }, + { + "budget": 0.94, + "threshold": -0.9610995054244995, + "operator": "ge", + "actual_fraction": 0.94, + "metric": 0.33, + "rescued": 3, + "harmed": 794, + "missed_rescue": 0, + "avoided_harm": 63, + "selected_both_correct": 321, + "selected_both_wrong": 10 + }, + { + "budget": 0.95, + "threshold": -0.9635111093521118, + "operator": "ge", + "actual_fraction": 0.95, + "metric": 0.32, + "rescued": 3, + "harmed": 806, + "missed_rescue": 0, + "avoided_harm": 51, + "selected_both_correct": 321, + "selected_both_wrong": 10 + }, + { + "budget": 0.96, + "threshold": -0.9648061990737915, + "operator": "ge", + "actual_fraction": 0.96, + "metric": 0.31333333333333335, + "rescued": 3, + "harmed": 814, + "missed_rescue": 0, + "avoided_harm": 43, + "selected_both_correct": 325, + "selected_both_wrong": 10 + }, + { + "budget": 0.97, + "threshold": -0.9674390554428101, + "operator": "ge", + "actual_fraction": 0.97, + "metric": 0.30416666666666664, + "rescued": 3, + "harmed": 825, + "missed_rescue": 0, + "avoided_harm": 32, + "selected_both_correct": 326, + "selected_both_wrong": 10 + }, + { + "budget": 0.98, + "threshold": -0.9724470973014832, + "operator": "ge", + "actual_fraction": 0.98, + "metric": 0.29583333333333334, + "rescued": 3, + "harmed": 835, + "missed_rescue": 0, + "avoided_harm": 22, + "selected_both_correct": 328, + "selected_both_wrong": 10 + }, + { + "budget": 0.99, + "threshold": -0.9774226546287537, + "operator": "ge", + "actual_fraction": 0.99, + "metric": 0.2866666666666667, + "rescued": 3, + "harmed": 846, + "missed_rescue": 0, + "avoided_harm": 11, + "selected_both_correct": 329, + "selected_both_wrong": 10 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.2775, + "rescued": 3, + "harmed": 857, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 330, + "selected_both_wrong": 10 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9016223679668622, + "rescued": 0, + "harmed": 0, + "missed_rescue": 106, + "avoided_harm": 3791, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.08698653779772178, + "metric": 0.8769416637901277, + "rescued": 50, + "harmed": 193, + "missed_rescue": 56, + "avoided_harm": 3598, + "selected_both_correct": 42, + "selected_both_wrong": 219 + }, + { + "budget": 0.02, + "threshold": -0.6014261841773987, + "operator": "ge", + "actual_fraction": 0.13910942354159475, + "metric": 0.8493268898860891, + "rescued": 66, + "harmed": 369, + "missed_rescue": 40, + "avoided_harm": 3422, + "selected_both_correct": 85, + "selected_both_wrong": 286 + }, + { + "budget": 0.03, + "threshold": -0.6401472687721252, + "operator": "ge", + "actual_fraction": 0.15947531929582326, + "metric": 0.8381083879875734, + "rescued": 74, + "harmed": 442, + "missed_rescue": 32, + "avoided_harm": 3349, + "selected_both_correct": 104, + "selected_both_wrong": 304 + }, + { + "budget": 0.04, + "threshold": -0.6884650588035583, + "operator": "ge", + "actual_fraction": 0.1886434242319641, + "metric": 0.8208491542975492, + "rescued": 81, + "harmed": 549, + "missed_rescue": 25, + "avoided_harm": 3242, + "selected_both_correct": 139, + "selected_both_wrong": 324 + }, + { + "budget": 0.05, + "threshold": -0.7208121418952942, + "operator": "ge", + "actual_fraction": 0.2143596824301001, + "metric": 0.8041076976182258, + "rescued": 84, + "harmed": 649, + "missed_rescue": 22, + "avoided_harm": 3142, + "selected_both_correct": 165, + "selected_both_wrong": 344 + }, + { + "budget": 0.06, + "threshold": -0.7578181624412537, + "operator": "ge", + "actual_fraction": 0.24525371073524335, + "metric": 0.7825336555056955, + "rescued": 88, + "harmed": 778, + "missed_rescue": 18, + "avoided_harm": 3013, + "selected_both_correct": 198, + "selected_both_wrong": 357 + }, + { + "budget": 0.07, + "threshold": -0.7790470719337463, + "operator": "ge", + "actual_fraction": 0.26734552985847426, + "metric": 0.767172937521574, + "rescued": 90, + "harmed": 869, + "missed_rescue": 16, + "avoided_harm": 2922, + "selected_both_correct": 221, + "selected_both_wrong": 369 + }, + { + "budget": 0.08, + "threshold": -0.7927228808403015, + "operator": "ge", + "actual_fraction": 0.2827062478425958, + "metric": 0.75716258198136, + "rescued": 92, + "harmed": 929, + "missed_rescue": 14, + "avoided_harm": 2862, + "selected_both_correct": 244, + "selected_both_wrong": 373 + }, + { + "budget": 0.09, + "threshold": -0.8049252033233643, + "operator": "ge", + "actual_fraction": 0.2999654815326199, + "metric": 0.7449085260614429, + "rescued": 93, + "harmed": 1001, + "missed_rescue": 13, + "avoided_harm": 2790, + "selected_both_correct": 264, + "selected_both_wrong": 380 + }, + { + "budget": 0.1, + "threshold": -0.80850750207901, + "operator": "ge", + "actual_fraction": 0.30531584397652745, + "metric": 0.7416292716603383, + "rescued": 93, + "harmed": 1020, + "missed_rescue": 13, + "avoided_harm": 2771, + "selected_both_correct": 274, + "selected_both_wrong": 382 + }, + { + "budget": 0.11, + "threshold": -0.819871723651886, + "operator": "ge", + "actual_fraction": 0.3258543320676562, + "metric": 0.7279944770452192, + "rescued": 96, + "harmed": 1102, + "missed_rescue": 10, + "avoided_harm": 2689, + "selected_both_correct": 302, + "selected_both_wrong": 388 + }, + { + "budget": 0.12, + "threshold": -0.8310170769691467, + "operator": "ge", + "actual_fraction": 0.35053503624439075, + "metric": 0.710735243355195, + "rescued": 99, + "harmed": 1205, + "missed_rescue": 7, + "avoided_harm": 2586, + "selected_both_correct": 332, + "selected_both_wrong": 395 + }, + { + "budget": 0.13, + "threshold": -0.8376906514167786, + "operator": "ge", + "actual_fraction": 0.36796686227131514, + "metric": 0.6976182257507767, + "rescued": 99, + "harmed": 1281, + "missed_rescue": 7, + "avoided_harm": 2510, + "selected_both_correct": 353, + "selected_both_wrong": 399 + }, + { + "budget": 0.14, + "threshold": -0.8419997692108154, + "operator": "ge", + "actual_fraction": 0.3798757335174318, + "metric": 0.6896789782533655, + "rescued": 99, + "harmed": 1327, + "missed_rescue": 7, + "avoided_harm": 2464, + "selected_both_correct": 374, + "selected_both_wrong": 401 + }, + { + "budget": 0.15, + "threshold": -0.8447275161743164, + "operator": "ge", + "actual_fraction": 0.3867794269934415, + "metric": 0.6855367621677597, + "rescued": 99, + "harmed": 1351, + "missed_rescue": 7, + "avoided_harm": 2440, + "selected_both_correct": 390, + "selected_both_wrong": 401 + }, + { + "budget": 0.16, + "threshold": -0.8506776690483093, + "operator": "ge", + "actual_fraction": 0.4050742147048671, + "metric": 0.6717293752157404, + "rescued": 100, + "harmed": 1432, + "missed_rescue": 6, + "avoided_harm": 2359, + "selected_both_correct": 411, + "selected_both_wrong": 404 + }, + { + "budget": 0.17, + "threshold": -0.8545639514923096, + "operator": "ge", + "actual_fraction": 0.4175008629616845, + "metric": 0.6632723507076286, + "rescued": 100, + "harmed": 1481, + "missed_rescue": 6, + "avoided_harm": 2310, + "selected_both_correct": 430, + "selected_both_wrong": 408 + }, + { + "budget": 0.18, + "threshold": -0.8578683137893677, + "operator": "ge", + "actual_fraction": 0.4290645495340007, + "metric": 0.6544701415257163, + "rescued": 101, + "harmed": 1533, + "missed_rescue": 5, + "avoided_harm": 2258, + "selected_both_correct": 443, + "selected_both_wrong": 409 + }, + { + "budget": 0.19, + "threshold": -0.86090087890625, + "operator": "ge", + "actual_fraction": 0.43993786675871593, + "metric": 0.6473938557128064, + "rescued": 101, + "harmed": 1574, + "missed_rescue": 5, + "avoided_harm": 2217, + "selected_both_correct": 462, + "selected_both_wrong": 412 + }, + { + "budget": 0.2, + "threshold": -0.8640955090522766, + "operator": "ge", + "actual_fraction": 0.4520193303417328, + "metric": 0.6389368312046945, + "rescued": 101, + "harmed": 1623, + "missed_rescue": 5, + "avoided_harm": 2168, + "selected_both_correct": 482, + "selected_both_wrong": 413 + }, + { + "budget": 0.21, + "threshold": -0.8664237260818481, + "operator": "ge", + "actual_fraction": 0.46047635484984467, + "metric": 0.6332412840869865, + "rescued": 101, + "harmed": 1656, + "missed_rescue": 5, + "avoided_harm": 2135, + "selected_both_correct": 496, + "selected_both_wrong": 415 + }, + { + "budget": 0.22, + "threshold": -0.8686786890029907, + "operator": "ge", + "actual_fraction": 0.46945115636865725, + "metric": 0.6272005522954781, + "rescued": 101, + "harmed": 1691, + "missed_rescue": 5, + "avoided_harm": 2100, + "selected_both_correct": 513, + "selected_both_wrong": 415 + }, + { + "budget": 0.23, + "threshold": -0.8709638118743896, + "operator": "ge", + "actual_fraction": 0.48066965826717295, + "metric": 0.6197790818087677, + "rescued": 101, + "harmed": 1734, + "missed_rescue": 5, + "avoided_harm": 2057, + "selected_both_correct": 534, + "selected_both_wrong": 416 + }, + { + "budget": 0.24, + "threshold": -0.8724074959754944, + "operator": "ge", + "actual_fraction": 0.48912668277528476, + "metric": 0.61425612702796, + "rescued": 101, + "harmed": 1766, + "missed_rescue": 5, + "avoided_harm": 2025, + "selected_both_correct": 550, + "selected_both_wrong": 417 + }, + { + "budget": 0.25, + "threshold": -0.8736594319343567, + "operator": "ge", + "actual_fraction": 0.49413186054539177, + "metric": 0.6113220573006558, + "rescued": 101, + "harmed": 1783, + "missed_rescue": 5, + "avoided_harm": 2008, + "selected_both_correct": 561, + "selected_both_wrong": 418 + }, + { + "budget": 0.26, + "threshold": -0.8752554059028625, + "operator": "ge", + "actual_fraction": 0.5005177770107008, + "metric": 0.6071798412150501, + "rescued": 101, + "harmed": 1807, + "missed_rescue": 5, + "avoided_harm": 1984, + "selected_both_correct": 574, + "selected_both_wrong": 418 + }, + { + "budget": 0.27, + "threshold": -0.8766024112701416, + "operator": "ge", + "actual_fraction": 0.5081118398343114, + "metric": 0.6020020711080428, + "rescued": 101, + "harmed": 1837, + "missed_rescue": 5, + "avoided_harm": 1954, + "selected_both_correct": 587, + "selected_both_wrong": 419 + }, + { + "budget": 0.28, + "threshold": -0.878311276435852, + "operator": "ge", + "actual_fraction": 0.5169140490162237, + "metric": 0.596479116327235, + "rescued": 102, + "harmed": 1870, + "missed_rescue": 4, + "avoided_harm": 1921, + "selected_both_correct": 603, + "selected_both_wrong": 420 + }, + { + "budget": 0.29, + "threshold": -0.880103588104248, + "operator": "ge", + "actual_fraction": 0.5277873662409389, + "metric": 0.5894028305143252, + "rescued": 102, + "harmed": 1911, + "missed_rescue": 4, + "avoided_harm": 1880, + "selected_both_correct": 624, + "selected_both_wrong": 421 + }, + { + "budget": 0.3, + "threshold": -0.8824952244758606, + "operator": "ge", + "actual_fraction": 0.5419399378667588, + "metric": 0.5788746979634104, + "rescued": 102, + "harmed": 1972, + "missed_rescue": 4, + "avoided_harm": 1819, + "selected_both_correct": 645, + "selected_both_wrong": 421 + }, + { + "budget": 0.31, + "threshold": -0.8831692337989807, + "operator": "ge", + "actual_fraction": 0.5453917846047636, + "metric": 0.5766309975837073, + "rescued": 102, + "harmed": 1985, + "missed_rescue": 4, + "avoided_harm": 1806, + "selected_both_correct": 651, + "selected_both_wrong": 422 + }, + { + "budget": 0.32, + "threshold": -0.8845130205154419, + "operator": "ge", + "actual_fraction": 0.5566102865032793, + "metric": 0.5704176734552986, + "rescued": 102, + "harmed": 2021, + "missed_rescue": 4, + "avoided_harm": 1770, + "selected_both_correct": 677, + "selected_both_wrong": 425 + }, + { + "budget": 0.33, + "threshold": -0.8855781555175781, + "operator": "ge", + "actual_fraction": 0.5633413876423887, + "metric": 0.5661028650327925, + "rescued": 102, + "harmed": 2046, + "missed_rescue": 4, + "avoided_harm": 1745, + "selected_both_correct": 691, + "selected_both_wrong": 425 + }, + { + "budget": 0.34, + "threshold": -0.8865919709205627, + "operator": "ge", + "actual_fraction": 0.5716258198136003, + "metric": 0.5607525025888851, + "rescued": 102, + "harmed": 2077, + "missed_rescue": 4, + "avoided_harm": 1714, + "selected_both_correct": 707, + "selected_both_wrong": 426 + }, + { + "budget": 0.35, + "threshold": -0.8877584338188171, + "operator": "ge", + "actual_fraction": 0.5819813600276148, + "metric": 0.5545391784604764, + "rescued": 103, + "harmed": 2114, + "missed_rescue": 3, + "avoided_harm": 1677, + "selected_both_correct": 728, + "selected_both_wrong": 427 + }, + { + "budget": 0.36, + "threshold": -0.8890330195426941, + "operator": "ge", + "actual_fraction": 0.5902657921988264, + "metric": 0.5493614083534691, + "rescued": 104, + "harmed": 2145, + "missed_rescue": 2, + "avoided_harm": 1646, + "selected_both_correct": 743, + "selected_both_wrong": 428 + }, + { + "budget": 0.37, + "threshold": -0.890850305557251, + "operator": "ge", + "actual_fraction": 0.6047635484984467, + "metric": 0.5383154987918537, + "rescued": 104, + "harmed": 2209, + "missed_rescue": 2, + "avoided_harm": 1582, + "selected_both_correct": 763, + "selected_both_wrong": 428 + }, + { + "budget": 0.38, + "threshold": -0.8916605710983276, + "operator": "ge", + "actual_fraction": 0.6113220573006558, + "metric": 0.5343458750431481, + "rescued": 104, + "harmed": 2232, + "missed_rescue": 2, + "avoided_harm": 1559, + "selected_both_correct": 778, + "selected_both_wrong": 428 + }, + { + "budget": 0.39, + "threshold": -0.8928589224815369, + "operator": "ge", + "actual_fraction": 0.6206420434932689, + "metric": 0.5279599585778392, + "rescued": 104, + "harmed": 2269, + "missed_rescue": 2, + "avoided_harm": 1522, + "selected_both_correct": 794, + "selected_both_wrong": 429 + }, + { + "budget": 0.4, + "threshold": -0.8940943479537964, + "operator": "ge", + "actual_fraction": 0.6289264756644805, + "metric": 0.5222644114601311, + "rescued": 104, + "harmed": 2302, + "missed_rescue": 2, + "avoided_harm": 1489, + "selected_both_correct": 809, + "selected_both_wrong": 429 + }, + { + "budget": 0.41, + "threshold": -0.8949463963508606, + "operator": "ge", + "actual_fraction": 0.6370383154987919, + "metric": 0.5169140490162237, + "rescued": 104, + "harmed": 2333, + "missed_rescue": 2, + "avoided_harm": 1458, + "selected_both_correct": 824, + "selected_both_wrong": 430 + }, + { + "budget": 0.42, + "threshold": -0.8960229158401489, + "operator": "ge", + "actual_fraction": 0.6435968243010011, + "metric": 0.5127718329306179, + "rescued": 104, + "harmed": 2357, + "missed_rescue": 2, + "avoided_harm": 1434, + "selected_both_correct": 836, + "selected_both_wrong": 432 + }, + { + "budget": 0.43, + "threshold": -0.8966536521911621, + "operator": "ge", + "actual_fraction": 0.6491197790818087, + "metric": 0.5082844321712116, + "rescued": 104, + "harmed": 2383, + "missed_rescue": 2, + "avoided_harm": 1408, + "selected_both_correct": 842, + "selected_both_wrong": 432 + }, + { + "budget": 0.44, + "threshold": -0.8975363373756409, + "operator": "ge", + "actual_fraction": 0.6558508802209182, + "metric": 0.5044874007594062, + "rescued": 104, + "harmed": 2405, + "missed_rescue": 2, + "avoided_harm": 1386, + "selected_both_correct": 859, + "selected_both_wrong": 432 + }, + { + "budget": 0.45, + "threshold": -0.8988122940063477, + "operator": "ge", + "actual_fraction": 0.6641353123921298, + "metric": 0.49896444597859857, + "rescued": 104, + "harmed": 2437, + "missed_rescue": 2, + "avoided_harm": 1354, + "selected_both_correct": 875, + "selected_both_wrong": 432 + }, + { + "budget": 0.46, + "threshold": -0.9000959396362305, + "operator": "ge", + "actual_fraction": 0.6739730755954436, + "metric": 0.4917155678287884, + "rescued": 104, + "harmed": 2479, + "missed_rescue": 2, + "avoided_harm": 1312, + "selected_both_correct": 890, + "selected_both_wrong": 432 + }, + { + "budget": 0.47, + "threshold": -0.9013051986694336, + "operator": "ge", + "actual_fraction": 0.6848463928201588, + "metric": 0.48412150500517775, + "rescued": 104, + "harmed": 2523, + "missed_rescue": 2, + "avoided_harm": 1268, + "selected_both_correct": 907, + "selected_both_wrong": 434 + }, + { + "budget": 0.48, + "threshold": -0.9029324054718018, + "operator": "ge", + "actual_fraction": 0.696410079392475, + "metric": 0.4775629962029686, + "rescued": 104, + "harmed": 2561, + "missed_rescue": 2, + "avoided_harm": 1230, + "selected_both_correct": 935, + "selected_both_wrong": 435 + }, + { + "budget": 0.49, + "threshold": -0.9036871790885925, + "operator": "ge", + "actual_fraction": 0.7026234035208837, + "metric": 0.47411114946496374, + "rescued": 104, + "harmed": 2581, + "missed_rescue": 2, + "avoided_harm": 1210, + "selected_both_correct": 949, + "selected_both_wrong": 437 + }, + { + "budget": 0.5, + "threshold": -0.9044621586799622, + "operator": "ge", + "actual_fraction": 0.7088367276492924, + "metric": 0.46996893337935797, + "rescued": 104, + "harmed": 2605, + "missed_rescue": 2, + "avoided_harm": 1186, + "selected_both_correct": 961, + "selected_both_wrong": 437 + }, + { + "budget": 0.51, + "threshold": -0.9054575562477112, + "operator": "ge", + "actual_fraction": 0.717121159820504, + "metric": 0.46496375560925096, + "rescued": 104, + "harmed": 2634, + "missed_rescue": 2, + "avoided_harm": 1157, + "selected_both_correct": 980, + "selected_both_wrong": 437 + }, + { + "budget": 0.52, + "threshold": -0.906682550907135, + "operator": "ge", + "actual_fraction": 0.7262685536762168, + "metric": 0.45875043148084227, + "rescued": 104, + "harmed": 2670, + "missed_rescue": 2, + "avoided_harm": 1121, + "selected_both_correct": 997, + "selected_both_wrong": 437 + }, + { + "budget": 0.53, + "threshold": -0.9072114825248718, + "operator": "ge", + "actual_fraction": 0.7300655850880221, + "metric": 0.4559889540904384, + "rescued": 104, + "harmed": 2686, + "missed_rescue": 2, + "avoided_harm": 1105, + "selected_both_correct": 1002, + "selected_both_wrong": 438 + }, + { + "budget": 0.54, + "threshold": -0.9083563089370728, + "operator": "ge", + "actual_fraction": 0.7381774249223334, + "metric": 0.4518467380048326, + "rescued": 105, + "harmed": 2711, + "missed_rescue": 1, + "avoided_harm": 1080, + "selected_both_correct": 1023, + "selected_both_wrong": 438 + }, + { + "budget": 0.55, + "threshold": -0.9095192551612854, + "operator": "ge", + "actual_fraction": 0.7487055574732482, + "metric": 0.4458060062133241, + "rescued": 105, + "harmed": 2746, + "missed_rescue": 1, + "avoided_harm": 1045, + "selected_both_correct": 1047, + "selected_both_wrong": 440 + }, + { + "budget": 0.56, + "threshold": -0.9106979370117188, + "operator": "ge", + "actual_fraction": 0.759233690024163, + "metric": 0.43924749741111496, + "rescued": 105, + "harmed": 2784, + "missed_rescue": 1, + "avoided_harm": 1007, + "selected_both_correct": 1069, + "selected_both_wrong": 441 + }, + { + "budget": 0.57, + "threshold": -0.9115362167358398, + "operator": "ge", + "actual_fraction": 0.767863306869175, + "metric": 0.4340697273041077, + "rescued": 105, + "harmed": 2814, + "missed_rescue": 1, + "avoided_harm": 977, + "selected_both_correct": 1089, + "selected_both_wrong": 441 + }, + { + "budget": 0.58, + "threshold": -0.9124338030815125, + "operator": "ge", + "actual_fraction": 0.7751121850189852, + "metric": 0.4301001035554021, + "rescued": 105, + "harmed": 2837, + "missed_rescue": 1, + "avoided_harm": 954, + "selected_both_correct": 1108, + "selected_both_wrong": 441 + }, + { + "budget": 0.59, + "threshold": -0.9134801626205444, + "operator": "ge", + "actual_fraction": 0.7830514325163963, + "metric": 0.42440455643769415, + "rescued": 105, + "harmed": 2870, + "missed_rescue": 1, + "avoided_harm": 921, + "selected_both_correct": 1121, + "selected_both_wrong": 441 + }, + { + "budget": 0.6, + "threshold": -0.9143747091293335, + "operator": "ge", + "actual_fraction": 0.7904729030031067, + "metric": 0.41939937866758714, + "rescued": 105, + "harmed": 2899, + "missed_rescue": 1, + "avoided_harm": 892, + "selected_both_correct": 1134, + "selected_both_wrong": 442 + }, + { + "budget": 0.61, + "threshold": -0.915804386138916, + "operator": "ge", + "actual_fraction": 0.8010010355540214, + "metric": 0.4133586468760787, + "rescued": 105, + "harmed": 2934, + "missed_rescue": 1, + "avoided_harm": 857, + "selected_both_correct": 1158, + "selected_both_wrong": 444 + }, + { + "budget": 0.62, + "threshold": -0.9165146350860596, + "operator": "ge", + "actual_fraction": 0.8063513979979289, + "metric": 0.41025198481187436, + "rescued": 105, + "harmed": 2952, + "missed_rescue": 1, + "avoided_harm": 839, + "selected_both_correct": 1171, + "selected_both_wrong": 444 + }, + { + "budget": 0.63, + "threshold": -0.9172822833061218, + "operator": "ge", + "actual_fraction": 0.8110113910942354, + "metric": 0.4062823610631688, + "rescued": 105, + "harmed": 2975, + "missed_rescue": 1, + "avoided_harm": 816, + "selected_both_correct": 1175, + "selected_both_wrong": 444 + }, + { + "budget": 0.64, + "threshold": -0.9181071519851685, + "operator": "ge", + "actual_fraction": 0.8172247152226441, + "metric": 0.40248532965136347, + "rescued": 105, + "harmed": 2997, + "missed_rescue": 1, + "avoided_harm": 794, + "selected_both_correct": 1188, + "selected_both_wrong": 445 + }, + { + "budget": 0.65, + "threshold": -0.9192029237747192, + "operator": "ge", + "actual_fraction": 0.824301001035554, + "metric": 0.3979979288919572, + "rescued": 105, + "harmed": 3023, + "missed_rescue": 1, + "avoided_harm": 768, + "selected_both_correct": 1202, + "selected_both_wrong": 446 + }, + { + "budget": 0.66, + "threshold": -0.9200543761253357, + "operator": "ge", + "actual_fraction": 0.829306178805661, + "metric": 0.39420089748015186, + "rescued": 105, + "harmed": 3045, + "missed_rescue": 1, + "avoided_harm": 746, + "selected_both_correct": 1209, + "selected_both_wrong": 446 + }, + { + "budget": 0.67, + "threshold": -0.9208071231842041, + "operator": "ge", + "actual_fraction": 0.8348291335864687, + "metric": 0.3898860890576458, + "rescued": 105, + "harmed": 3070, + "missed_rescue": 1, + "avoided_harm": 721, + "selected_both_correct": 1216, + "selected_both_wrong": 446 + }, + { + "budget": 0.68, + "threshold": -0.9215553998947144, + "operator": "ge", + "actual_fraction": 0.840006903693476, + "metric": 0.3867794269934415, + "rescued": 105, + "harmed": 3088, + "missed_rescue": 1, + "avoided_harm": 703, + "selected_both_correct": 1228, + "selected_both_wrong": 446 + }, + { + "budget": 0.69, + "threshold": -0.9220835566520691, + "operator": "ge", + "actual_fraction": 0.8443217121159821, + "metric": 0.38401794960303764, + "rescued": 105, + "harmed": 3104, + "missed_rescue": 1, + "avoided_harm": 687, + "selected_both_correct": 1236, + "selected_both_wrong": 447 + }, + { + "budget": 0.7, + "threshold": -0.9232237935066223, + "operator": "ge", + "actual_fraction": 0.852088367276493, + "metric": 0.37849499482222987, + "rescued": 105, + "harmed": 3136, + "missed_rescue": 1, + "avoided_harm": 655, + "selected_both_correct": 1247, + "selected_both_wrong": 449 + }, + { + "budget": 0.71, + "threshold": -0.9243364334106445, + "operator": "ge", + "actual_fraction": 0.8598550224370038, + "metric": 0.37279944770452195, + "rescued": 105, + "harmed": 3169, + "missed_rescue": 1, + "avoided_harm": 622, + "selected_both_correct": 1257, + "selected_both_wrong": 451 + }, + { + "budget": 0.72, + "threshold": -0.9252541065216064, + "operator": "ge", + "actual_fraction": 0.8641698308595098, + "metric": 0.37003797031411806, + "rescued": 105, + "harmed": 3185, + "missed_rescue": 1, + "avoided_harm": 606, + "selected_both_correct": 1266, + "selected_both_wrong": 451 + }, + { + "budget": 0.73, + "threshold": -0.925781786441803, + "operator": "ge", + "actual_fraction": 0.8681394546082154, + "metric": 0.3674490852606144, + "rescued": 105, + "harmed": 3200, + "missed_rescue": 1, + "avoided_harm": 591, + "selected_both_correct": 1274, + "selected_both_wrong": 451 + }, + { + "budget": 0.74, + "threshold": -0.9274963140487671, + "operator": "ge", + "actual_fraction": 0.877114256127028, + "metric": 0.36123576113220573, + "rescued": 105, + "harmed": 3236, + "missed_rescue": 1, + "avoided_harm": 555, + "selected_both_correct": 1290, + "selected_both_wrong": 451 + }, + { + "budget": 0.75, + "threshold": -0.9286423325538635, + "operator": "ge", + "actual_fraction": 0.8828098032447359, + "metric": 0.35726613738350016, + "rescued": 105, + "harmed": 3259, + "missed_rescue": 1, + "avoided_harm": 532, + "selected_both_correct": 1299, + "selected_both_wrong": 452 + }, + { + "budget": 0.76, + "threshold": -0.9301291704177856, + "operator": "ge", + "actual_fraction": 0.8900586813945461, + "metric": 0.3520883672764929, + "rescued": 105, + "harmed": 3289, + "missed_rescue": 1, + "avoided_harm": 502, + "selected_both_correct": 1310, + "selected_both_wrong": 453 + }, + { + "budget": 0.77, + "threshold": -0.9318777918815613, + "operator": "ge", + "actual_fraction": 0.8979979288919572, + "metric": 0.34673800483258543, + "rescued": 105, + "harmed": 3320, + "missed_rescue": 1, + "avoided_harm": 471, + "selected_both_correct": 1325, + "selected_both_wrong": 453 + }, + { + "budget": 0.78, + "threshold": -0.9332055449485779, + "operator": "ge", + "actual_fraction": 0.9052468070417673, + "metric": 0.34173282706247843, + "rescued": 105, + "harmed": 3349, + "missed_rescue": 1, + "avoided_harm": 442, + "selected_both_correct": 1338, + "selected_both_wrong": 453 + }, + { + "budget": 0.79, + "threshold": -0.9344666600227356, + "operator": "ge", + "actual_fraction": 0.9100793924749742, + "metric": 0.3379357956506731, + "rescued": 105, + "harmed": 3371, + "missed_rescue": 1, + "avoided_harm": 420, + "selected_both_correct": 1344, + "selected_both_wrong": 453 + }, + { + "budget": 0.8, + "threshold": -0.9362465143203735, + "operator": "ge", + "actual_fraction": 0.917155678287884, + "metric": 0.33224024853296513, + "rescued": 105, + "harmed": 3404, + "missed_rescue": 1, + "avoided_harm": 387, + "selected_both_correct": 1351, + "selected_both_wrong": 454 + }, + { + "budget": 0.81, + "threshold": -0.937842071056366, + "operator": "ge", + "actual_fraction": 0.9237141870900932, + "metric": 0.3274076630997584, + "rescued": 105, + "harmed": 3432, + "missed_rescue": 1, + "avoided_harm": 359, + "selected_both_correct": 1361, + "selected_both_wrong": 454 + }, + { + "budget": 0.82, + "threshold": -0.9384457468986511, + "operator": "ge", + "actual_fraction": 0.9249223334483949, + "metric": 0.3267172937521574, + "rescued": 105, + "harmed": 3436, + "missed_rescue": 1, + "avoided_harm": 355, + "selected_both_correct": 1363, + "selected_both_wrong": 455 + }, + { + "budget": 0.83, + "threshold": -0.9396296143531799, + "operator": "ge", + "actual_fraction": 0.927166033828098, + "metric": 0.324991370383155, + "rescued": 105, + "harmed": 3446, + "missed_rescue": 1, + "avoided_harm": 345, + "selected_both_correct": 1365, + "selected_both_wrong": 456 + }, + { + "budget": 0.84, + "threshold": -0.9407488703727722, + "operator": "ge", + "actual_fraction": 0.9316534345875043, + "metric": 0.3211943389713497, + "rescued": 105, + "harmed": 3468, + "missed_rescue": 1, + "avoided_harm": 323, + "selected_both_correct": 1369, + "selected_both_wrong": 456 + }, + { + "budget": 0.85, + "threshold": -0.942738950252533, + "operator": "ge", + "actual_fraction": 0.9397652744218157, + "metric": 0.31429064549534, + "rescued": 105, + "harmed": 3508, + "missed_rescue": 1, + "avoided_harm": 283, + "selected_both_correct": 1376, + "selected_both_wrong": 456 + }, + { + "budget": 0.86, + "threshold": -0.9438527226448059, + "operator": "gt", + "actual_fraction": 0.9432171211598205, + "metric": 0.31239212978943737, + "rescued": 105, + "harmed": 3519, + "missed_rescue": 1, + "avoided_harm": 272, + "selected_both_correct": 1384, + "selected_both_wrong": 457 + }, + { + "budget": 0.87, + "threshold": -0.9450756907463074, + "operator": "ge", + "actual_fraction": 0.9456334138764239, + "metric": 0.3106662064204349, + "rescued": 105, + "harmed": 3529, + "missed_rescue": 1, + "avoided_harm": 262, + "selected_both_correct": 1388, + "selected_both_wrong": 457 + }, + { + "budget": 0.88, + "threshold": -0.9481598138809204, + "operator": "ge", + "actual_fraction": 0.9539178460476355, + "metric": 0.3044528822920262, + "rescued": 105, + "harmed": 3565, + "missed_rescue": 1, + "avoided_harm": 226, + "selected_both_correct": 1398, + "selected_both_wrong": 459 + }, + { + "budget": 0.89, + "threshold": -0.9500836730003357, + "operator": "ge", + "actual_fraction": 0.9606489471867449, + "metric": 0.29858474283741804, + "rescued": 105, + "harmed": 3599, + "missed_rescue": 1, + "avoided_harm": 192, + "selected_both_correct": 1402, + "selected_both_wrong": 460 + }, + { + "budget": 0.9, + "threshold": -0.9519118666648865, + "operator": "ge", + "actual_fraction": 0.9647911632723507, + "metric": 0.2956506731101139, + "rescued": 105, + "harmed": 3616, + "missed_rescue": 1, + "avoided_harm": 175, + "selected_both_correct": 1408, + "selected_both_wrong": 461 + }, + { + "budget": 0.91, + "threshold": -0.9543898701667786, + "operator": "ge", + "actual_fraction": 0.9697963410424577, + "metric": 0.2911632723507076, + "rescued": 105, + "harmed": 3642, + "missed_rescue": 1, + "avoided_harm": 149, + "selected_both_correct": 1410, + "selected_both_wrong": 462 + }, + { + "budget": 0.92, + "threshold": -0.9570822715759277, + "operator": "ge", + "actual_fraction": 0.9760096651708664, + "metric": 0.28633068691750085, + "rescued": 106, + "harmed": 3671, + "missed_rescue": 0, + "avoided_harm": 120, + "selected_both_correct": 1416, + "selected_both_wrong": 462 + }, + { + "budget": 0.93, + "threshold": -0.9593446254730225, + "operator": "ge", + "actual_fraction": 0.9801518812564722, + "metric": 0.2827062478425958, + "rescued": 106, + "harmed": 3692, + "missed_rescue": 0, + "avoided_harm": 99, + "selected_both_correct": 1419, + "selected_both_wrong": 462 + }, + { + "budget": 0.94, + "threshold": -0.9610995054244995, + "operator": "ge", + "actual_fraction": 0.9817052122885744, + "metric": 0.28132550914739385, + "rescued": 106, + "harmed": 3700, + "missed_rescue": 0, + "avoided_harm": 91, + "selected_both_correct": 1420, + "selected_both_wrong": 462 + }, + { + "budget": 0.95, + "threshold": -0.9635111093521118, + "operator": "ge", + "actual_fraction": 0.9853296513634795, + "metric": 0.2787366240938902, + "rescued": 106, + "harmed": 3715, + "missed_rescue": 0, + "avoided_harm": 76, + "selected_both_correct": 1426, + "selected_both_wrong": 462 + }, + { + "budget": 0.96, + "threshold": -0.9648061990737915, + "operator": "ge", + "actual_fraction": 0.9875733517431826, + "metric": 0.27666551605108736, + "rescued": 106, + "harmed": 3727, + "missed_rescue": 0, + "avoided_harm": 64, + "selected_both_correct": 1427, + "selected_both_wrong": 462 + }, + { + "budget": 0.97, + "threshold": -0.9674390554428101, + "operator": "ge", + "actual_fraction": 0.9906800138073869, + "metric": 0.27390403866068347, + "rescued": 106, + "harmed": 3743, + "missed_rescue": 0, + "avoided_harm": 48, + "selected_both_correct": 1429, + "selected_both_wrong": 462 + }, + { + "budget": 0.98, + "threshold": -0.9724470973014832, + "operator": "ge", + "actual_fraction": 0.9937866758715913, + "metric": 0.2711425612702796, + "rescued": 106, + "harmed": 3759, + "missed_rescue": 0, + "avoided_harm": 32, + "selected_both_correct": 1430, + "selected_both_wrong": 463 + }, + { + "budget": 0.99, + "threshold": -0.9774226546287537, + "operator": "ge", + "actual_fraction": 0.9965481532619952, + "metric": 0.26889886089057646, + "rescued": 106, + "harmed": 3772, + "missed_rescue": 0, + "avoided_harm": 19, + "selected_both_correct": 1432, + "selected_both_wrong": 464 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.26561960648947186, + "rescued": 106, + "harmed": 3791, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 1433, + "selected_both_wrong": 464 + } + ] + }, + "learned": { + "selected_budget_index": 3, + "validation_area_0_25": 0.97075, + "validation_point": { + "budget": 0.03, + "threshold": -1.5600132942199707, + "operator": "ge", + "actual_fraction": 0.03, + "metric": 0.99, + "rescued": 2, + "harmed": 1, + "missed_rescue": 1, + "avoided_harm": 856, + "selected_both_correct": 29, + "selected_both_wrong": 4 + }, + "test_point": { + "budget": 0.03, + "threshold": -1.5600132942199707, + "operator": "ge", + "actual_fraction": 0.04487400759406282, + "metric": 0.8979979288919572, + "rescued": 9, + "harmed": 30, + "missed_rescue": 97, + "avoided_harm": 3761, + "selected_both_correct": 171, + "selected_both_wrong": 50 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9891666666666666, + "rescued": 0, + "harmed": 0, + "missed_rescue": 3, + "avoided_harm": 857, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 0.5851550102233887, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9891666666666666, + "rescued": 0, + "harmed": 0, + "missed_rescue": 3, + "avoided_harm": 857, + "selected_both_correct": 9, + "selected_both_wrong": 3 + }, + { + "budget": 0.02, + "threshold": -0.7202239036560059, + "operator": "ge", + "actual_fraction": 0.02, + "metric": 0.9891666666666666, + "rescued": 1, + "harmed": 1, + "missed_rescue": 2, + "avoided_harm": 856, + "selected_both_correct": 19, + "selected_both_wrong": 3 + }, + { + "budget": 0.03, + "threshold": -1.5600132942199707, + "operator": "ge", + "actual_fraction": 0.03, + "metric": 0.99, + "rescued": 2, + "harmed": 1, + "missed_rescue": 1, + "avoided_harm": 856, + "selected_both_correct": 29, + "selected_both_wrong": 4 + }, + { + "budget": 0.04, + "threshold": -2.136852741241455, + "operator": "ge", + "actual_fraction": 0.04, + "metric": 0.99, + "rescued": 2, + "harmed": 1, + "missed_rescue": 1, + "avoided_harm": 856, + "selected_both_correct": 40, + "selected_both_wrong": 5 + }, + { + "budget": 0.05, + "threshold": -3.080942392349243, + "operator": "gt", + "actual_fraction": 0.049166666666666664, + "metric": 0.9891666666666666, + "rescued": 2, + "harmed": 2, + "missed_rescue": 1, + "avoided_harm": 855, + "selected_both_correct": 50, + "selected_both_wrong": 5 + }, + { + "budget": 0.06, + "threshold": -3.58967924118042, + "operator": "ge", + "actual_fraction": 0.06, + "metric": 0.9891666666666666, + "rescued": 2, + "harmed": 2, + "missed_rescue": 1, + "avoided_harm": 855, + "selected_both_correct": 63, + "selected_both_wrong": 5 + }, + { + "budget": 0.07, + "threshold": -4.303406715393066, + "operator": "ge", + "actual_fraction": 0.07, + "metric": 0.9866666666666667, + "rescued": 2, + "harmed": 5, + "missed_rescue": 1, + "avoided_harm": 852, + "selected_both_correct": 72, + "selected_both_wrong": 5 + }, + { + "budget": 0.08, + "threshold": -4.7087507247924805, + "operator": "ge", + "actual_fraction": 0.08, + "metric": 0.9858333333333333, + "rescued": 3, + "harmed": 7, + "missed_rescue": 0, + "avoided_harm": 850, + "selected_both_correct": 81, + "selected_both_wrong": 5 + }, + { + "budget": 0.09, + "threshold": -5.179437637329102, + "operator": "ge", + "actual_fraction": 0.09, + "metric": 0.985, + "rescued": 3, + "harmed": 8, + "missed_rescue": 0, + "avoided_harm": 849, + "selected_both_correct": 92, + "selected_both_wrong": 5 + }, + { + "budget": 0.1, + "threshold": -5.7383904457092285, + "operator": "ge", + "actual_fraction": 0.1, + "metric": 0.9833333333333333, + "rescued": 3, + "harmed": 10, + "missed_rescue": 0, + "avoided_harm": 847, + "selected_both_correct": 102, + "selected_both_wrong": 5 + }, + { + "budget": 0.11, + "threshold": -6.130518913269043, + "operator": "ge", + "actual_fraction": 0.11, + "metric": 0.9825, + "rescued": 3, + "harmed": 11, + "missed_rescue": 0, + "avoided_harm": 846, + "selected_both_correct": 113, + "selected_both_wrong": 5 + }, + { + "budget": 0.12, + "threshold": -6.5530104637146, + "operator": "ge", + "actual_fraction": 0.12, + "metric": 0.9808333333333333, + "rescued": 3, + "harmed": 13, + "missed_rescue": 0, + "avoided_harm": 844, + "selected_both_correct": 123, + "selected_both_wrong": 5 + }, + { + "budget": 0.13, + "threshold": -6.861036777496338, + "operator": "ge", + "actual_fraction": 0.13, + "metric": 0.9783333333333334, + "rescued": 3, + "harmed": 16, + "missed_rescue": 0, + "avoided_harm": 841, + "selected_both_correct": 132, + "selected_both_wrong": 5 + }, + { + "budget": 0.14, + "threshold": -7.096773147583008, + "operator": "ge", + "actual_fraction": 0.14, + "metric": 0.9766666666666667, + "rescued": 3, + "harmed": 18, + "missed_rescue": 0, + "avoided_harm": 839, + "selected_both_correct": 142, + "selected_both_wrong": 5 + }, + { + "budget": 0.15, + "threshold": -7.499796390533447, + "operator": "ge", + "actual_fraction": 0.15, + "metric": 0.9716666666666667, + "rescued": 3, + "harmed": 24, + "missed_rescue": 0, + "avoided_harm": 833, + "selected_both_correct": 148, + "selected_both_wrong": 5 + }, + { + "budget": 0.16, + "threshold": -7.857483386993408, + "operator": "ge", + "actual_fraction": 0.16, + "metric": 0.9691666666666666, + "rescued": 3, + "harmed": 27, + "missed_rescue": 0, + "avoided_harm": 830, + "selected_both_correct": 157, + "selected_both_wrong": 5 + }, + { + "budget": 0.17, + "threshold": -8.096627235412598, + "operator": "ge", + "actual_fraction": 0.17, + "metric": 0.9666666666666667, + "rescued": 3, + "harmed": 30, + "missed_rescue": 0, + "avoided_harm": 827, + "selected_both_correct": 166, + "selected_both_wrong": 5 + }, + { + "budget": 0.18, + "threshold": -8.421232223510742, + "operator": "ge", + "actual_fraction": 0.18, + "metric": 0.9608333333333333, + "rescued": 3, + "harmed": 37, + "missed_rescue": 0, + "avoided_harm": 820, + "selected_both_correct": 171, + "selected_both_wrong": 5 + }, + { + "budget": 0.19, + "threshold": -8.759540557861328, + "operator": "ge", + "actual_fraction": 0.19, + "metric": 0.9558333333333333, + "rescued": 3, + "harmed": 43, + "missed_rescue": 0, + "avoided_harm": 814, + "selected_both_correct": 177, + "selected_both_wrong": 5 + }, + { + "budget": 0.2, + "threshold": -9.157719612121582, + "operator": "ge", + "actual_fraction": 0.2, + "metric": 0.9516666666666667, + "rescued": 3, + "harmed": 48, + "missed_rescue": 0, + "avoided_harm": 809, + "selected_both_correct": 184, + "selected_both_wrong": 5 + }, + { + "budget": 0.21, + "threshold": -9.493932723999023, + "operator": "ge", + "actual_fraction": 0.21, + "metric": 0.945, + "rescued": 3, + "harmed": 56, + "missed_rescue": 0, + "avoided_harm": 801, + "selected_both_correct": 188, + "selected_both_wrong": 5 + }, + { + "budget": 0.22, + "threshold": -9.7905912399292, + "operator": "ge", + "actual_fraction": 0.22, + "metric": 0.9408333333333333, + "rescued": 3, + "harmed": 61, + "missed_rescue": 0, + "avoided_harm": 796, + "selected_both_correct": 195, + "selected_both_wrong": 5 + }, + { + "budget": 0.23, + "threshold": -10.482229232788086, + "operator": "ge", + "actual_fraction": 0.23, + "metric": 0.9341666666666667, + "rescued": 3, + "harmed": 69, + "missed_rescue": 0, + "avoided_harm": 788, + "selected_both_correct": 199, + "selected_both_wrong": 5 + }, + { + "budget": 0.24, + "threshold": -10.810736656188965, + "operator": "ge", + "actual_fraction": 0.24, + "metric": 0.9308333333333333, + "rescued": 3, + "harmed": 73, + "missed_rescue": 0, + "avoided_harm": 784, + "selected_both_correct": 207, + "selected_both_wrong": 5 + }, + { + "budget": 0.25, + "threshold": -11.235258102416992, + "operator": "ge", + "actual_fraction": 0.25, + "metric": 0.9266666666666666, + "rescued": 3, + "harmed": 78, + "missed_rescue": 0, + "avoided_harm": 779, + "selected_both_correct": 213, + "selected_both_wrong": 6 + }, + { + "budget": 0.26, + "threshold": -11.582498550415039, + "operator": "ge", + "actual_fraction": 0.26, + "metric": 0.9216666666666666, + "rescued": 3, + "harmed": 84, + "missed_rescue": 0, + "avoided_harm": 773, + "selected_both_correct": 219, + "selected_both_wrong": 6 + }, + { + "budget": 0.27, + "threshold": -11.94222640991211, + "operator": "ge", + "actual_fraction": 0.27, + "metric": 0.915, + "rescued": 3, + "harmed": 92, + "missed_rescue": 0, + "avoided_harm": 765, + "selected_both_correct": 223, + "selected_both_wrong": 6 + }, + { + "budget": 0.28, + "threshold": -12.258855819702148, + "operator": "ge", + "actual_fraction": 0.28, + "metric": 0.9091666666666667, + "rescued": 3, + "harmed": 99, + "missed_rescue": 0, + "avoided_harm": 758, + "selected_both_correct": 228, + "selected_both_wrong": 6 + }, + { + "budget": 0.29, + "threshold": -12.57064437866211, + "operator": "ge", + "actual_fraction": 0.29, + "metric": 0.9025, + "rescued": 3, + "harmed": 107, + "missed_rescue": 0, + "avoided_harm": 750, + "selected_both_correct": 232, + "selected_both_wrong": 6 + }, + { + "budget": 0.3, + "threshold": -12.848037719726562, + "operator": "ge", + "actual_fraction": 0.3, + "metric": 0.8958333333333334, + "rescued": 3, + "harmed": 115, + "missed_rescue": 0, + "avoided_harm": 742, + "selected_both_correct": 236, + "selected_both_wrong": 6 + }, + { + "budget": 0.31, + "threshold": -12.99221420288086, + "operator": "ge", + "actual_fraction": 0.31, + "metric": 0.8866666666666667, + "rescued": 3, + "harmed": 126, + "missed_rescue": 0, + "avoided_harm": 731, + "selected_both_correct": 237, + "selected_both_wrong": 6 + }, + { + "budget": 0.32, + "threshold": -13.233619689941406, + "operator": "ge", + "actual_fraction": 0.32, + "metric": 0.8791666666666667, + "rescued": 3, + "harmed": 135, + "missed_rescue": 0, + "avoided_harm": 722, + "selected_both_correct": 240, + "selected_both_wrong": 6 + }, + { + "budget": 0.33, + "threshold": -13.389532089233398, + "operator": "ge", + "actual_fraction": 0.33, + "metric": 0.87, + "rescued": 3, + "harmed": 146, + "missed_rescue": 0, + "avoided_harm": 711, + "selected_both_correct": 241, + "selected_both_wrong": 6 + }, + { + "budget": 0.34, + "threshold": -13.660714149475098, + "operator": "ge", + "actual_fraction": 0.34, + "metric": 0.8658333333333333, + "rescued": 3, + "harmed": 151, + "missed_rescue": 0, + "avoided_harm": 706, + "selected_both_correct": 248, + "selected_both_wrong": 6 + }, + { + "budget": 0.35, + "threshold": -13.987701416015625, + "operator": "ge", + "actual_fraction": 0.35, + "metric": 0.8591666666666666, + "rescued": 3, + "harmed": 159, + "missed_rescue": 0, + "avoided_harm": 698, + "selected_both_correct": 252, + "selected_both_wrong": 6 + }, + { + "budget": 0.36, + "threshold": -14.314142227172852, + "operator": "ge", + "actual_fraction": 0.36, + "metric": 0.8516666666666667, + "rescued": 3, + "harmed": 168, + "missed_rescue": 0, + "avoided_harm": 689, + "selected_both_correct": 254, + "selected_both_wrong": 7 + }, + { + "budget": 0.37, + "threshold": -14.651002883911133, + "operator": "ge", + "actual_fraction": 0.37, + "metric": 0.845, + "rescued": 3, + "harmed": 176, + "missed_rescue": 0, + "avoided_harm": 681, + "selected_both_correct": 258, + "selected_both_wrong": 7 + }, + { + "budget": 0.38, + "threshold": -14.930255889892578, + "operator": "ge", + "actual_fraction": 0.38, + "metric": 0.8375, + "rescued": 3, + "harmed": 185, + "missed_rescue": 0, + "avoided_harm": 672, + "selected_both_correct": 261, + "selected_both_wrong": 7 + }, + { + "budget": 0.39, + "threshold": -15.176605224609375, + "operator": "ge", + "actual_fraction": 0.39, + "metric": 0.8308333333333333, + "rescued": 3, + "harmed": 193, + "missed_rescue": 0, + "avoided_harm": 664, + "selected_both_correct": 265, + "selected_both_wrong": 7 + }, + { + "budget": 0.4, + "threshold": -15.31328010559082, + "operator": "ge", + "actual_fraction": 0.4, + "metric": 0.8216666666666667, + "rescued": 3, + "harmed": 204, + "missed_rescue": 0, + "avoided_harm": 653, + "selected_both_correct": 266, + "selected_both_wrong": 7 + }, + { + "budget": 0.41, + "threshold": -15.492185592651367, + "operator": "ge", + "actual_fraction": 0.4091666666666667, + "metric": 0.8158333333333333, + "rescued": 3, + "harmed": 211, + "missed_rescue": 0, + "avoided_harm": 646, + "selected_both_correct": 270, + "selected_both_wrong": 7 + }, + { + "budget": 0.42, + "threshold": -15.74117374420166, + "operator": "ge", + "actual_fraction": 0.42, + "metric": 0.8083333333333333, + "rescued": 3, + "harmed": 220, + "missed_rescue": 0, + "avoided_harm": 637, + "selected_both_correct": 274, + "selected_both_wrong": 7 + }, + { + "budget": 0.43, + "threshold": -16.078542709350586, + "operator": "ge", + "actual_fraction": 0.43, + "metric": 0.8, + "rescued": 3, + "harmed": 230, + "missed_rescue": 0, + "avoided_harm": 627, + "selected_both_correct": 276, + "selected_both_wrong": 7 + }, + { + "budget": 0.44, + "threshold": -16.330650329589844, + "operator": "ge", + "actual_fraction": 0.44, + "metric": 0.7916666666666666, + "rescued": 3, + "harmed": 240, + "missed_rescue": 0, + "avoided_harm": 617, + "selected_both_correct": 278, + "selected_both_wrong": 7 + }, + { + "budget": 0.45, + "threshold": -16.650772094726562, + "operator": "ge", + "actual_fraction": 0.45, + "metric": 0.7841666666666667, + "rescued": 3, + "harmed": 249, + "missed_rescue": 0, + "avoided_harm": 608, + "selected_both_correct": 281, + "selected_both_wrong": 7 + }, + { + "budget": 0.46, + "threshold": -16.817930221557617, + "operator": "ge", + "actual_fraction": 0.46, + "metric": 0.775, + "rescued": 3, + "harmed": 260, + "missed_rescue": 0, + "avoided_harm": 597, + "selected_both_correct": 282, + "selected_both_wrong": 7 + }, + { + "budget": 0.47, + "threshold": -17.027217864990234, + "operator": "ge", + "actual_fraction": 0.47, + "metric": 0.7675, + "rescued": 3, + "harmed": 269, + "missed_rescue": 0, + "avoided_harm": 588, + "selected_both_correct": 285, + "selected_both_wrong": 7 + }, + { + "budget": 0.48, + "threshold": -17.246978759765625, + "operator": "ge", + "actual_fraction": 0.48, + "metric": 0.7591666666666667, + "rescued": 3, + "harmed": 279, + "missed_rescue": 0, + "avoided_harm": 578, + "selected_both_correct": 287, + "selected_both_wrong": 7 + }, + { + "budget": 0.49, + "threshold": -17.445709228515625, + "operator": "ge", + "actual_fraction": 0.49, + "metric": 0.75, + "rescued": 3, + "harmed": 290, + "missed_rescue": 0, + "avoided_harm": 567, + "selected_both_correct": 288, + "selected_both_wrong": 7 + }, + { + "budget": 0.5, + "threshold": -17.677627563476562, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.74, + "rescued": 3, + "harmed": 302, + "missed_rescue": 0, + "avoided_harm": 555, + "selected_both_correct": 288, + "selected_both_wrong": 7 + }, + { + "budget": 0.51, + "threshold": -17.777769088745117, + "operator": "ge", + "actual_fraction": 0.51, + "metric": 0.7333333333333333, + "rescued": 3, + "harmed": 310, + "missed_rescue": 0, + "avoided_harm": 547, + "selected_both_correct": 292, + "selected_both_wrong": 7 + }, + { + "budget": 0.52, + "threshold": -17.976903915405273, + "operator": "ge", + "actual_fraction": 0.52, + "metric": 0.7241666666666666, + "rescued": 3, + "harmed": 321, + "missed_rescue": 0, + "avoided_harm": 536, + "selected_both_correct": 293, + "selected_both_wrong": 7 + }, + { + "budget": 0.53, + "threshold": -18.135372161865234, + "operator": "ge", + "actual_fraction": 0.53, + "metric": 0.7158333333333333, + "rescued": 3, + "harmed": 331, + "missed_rescue": 0, + "avoided_harm": 526, + "selected_both_correct": 295, + "selected_both_wrong": 7 + }, + { + "budget": 0.54, + "threshold": -18.381885528564453, + "operator": "ge", + "actual_fraction": 0.54, + "metric": 0.7083333333333334, + "rescued": 3, + "harmed": 340, + "missed_rescue": 0, + "avoided_harm": 517, + "selected_both_correct": 297, + "selected_both_wrong": 8 + }, + { + "budget": 0.55, + "threshold": -18.523502349853516, + "operator": "ge", + "actual_fraction": 0.55, + "metric": 0.7, + "rescued": 3, + "harmed": 350, + "missed_rescue": 0, + "avoided_harm": 507, + "selected_both_correct": 299, + "selected_both_wrong": 8 + }, + { + "budget": 0.56, + "threshold": -18.71038246154785, + "operator": "ge", + "actual_fraction": 0.56, + "metric": 0.6908333333333333, + "rescued": 3, + "harmed": 361, + "missed_rescue": 0, + "avoided_harm": 496, + "selected_both_correct": 300, + "selected_both_wrong": 8 + }, + { + "budget": 0.57, + "threshold": -18.846454620361328, + "operator": "ge", + "actual_fraction": 0.5691666666666667, + "metric": 0.6825, + "rescued": 3, + "harmed": 371, + "missed_rescue": 0, + "avoided_harm": 486, + "selected_both_correct": 301, + "selected_both_wrong": 8 + }, + { + "budget": 0.58, + "threshold": -19.10778045654297, + "operator": "ge", + "actual_fraction": 0.58, + "metric": 0.6725, + "rescued": 3, + "harmed": 383, + "missed_rescue": 0, + "avoided_harm": 474, + "selected_both_correct": 302, + "selected_both_wrong": 8 + }, + { + "budget": 0.59, + "threshold": -19.334182739257812, + "operator": "ge", + "actual_fraction": 0.59, + "metric": 0.665, + "rescued": 3, + "harmed": 392, + "missed_rescue": 0, + "avoided_harm": 465, + "selected_both_correct": 304, + "selected_both_wrong": 9 + }, + { + "budget": 0.6, + "threshold": -19.49287986755371, + "operator": "ge", + "actual_fraction": 0.6, + "metric": 0.655, + "rescued": 3, + "harmed": 404, + "missed_rescue": 0, + "avoided_harm": 453, + "selected_both_correct": 304, + "selected_both_wrong": 9 + }, + { + "budget": 0.61, + "threshold": -19.699918746948242, + "operator": "ge", + "actual_fraction": 0.61, + "metric": 0.6466666666666666, + "rescued": 3, + "harmed": 414, + "missed_rescue": 0, + "avoided_harm": 443, + "selected_both_correct": 306, + "selected_both_wrong": 9 + }, + { + "budget": 0.62, + "threshold": -19.920011520385742, + "operator": "ge", + "actual_fraction": 0.62, + "metric": 0.6375, + "rescued": 3, + "harmed": 425, + "missed_rescue": 0, + "avoided_harm": 432, + "selected_both_correct": 307, + "selected_both_wrong": 9 + }, + { + "budget": 0.63, + "threshold": -20.140621185302734, + "operator": "ge", + "actual_fraction": 0.63, + "metric": 0.6283333333333333, + "rescued": 3, + "harmed": 436, + "missed_rescue": 0, + "avoided_harm": 421, + "selected_both_correct": 308, + "selected_both_wrong": 9 + }, + { + "budget": 0.64, + "threshold": -20.324838638305664, + "operator": "ge", + "actual_fraction": 0.64, + "metric": 0.62, + "rescued": 3, + "harmed": 446, + "missed_rescue": 0, + "avoided_harm": 411, + "selected_both_correct": 309, + "selected_both_wrong": 10 + }, + { + "budget": 0.65, + "threshold": -20.663583755493164, + "operator": "ge", + "actual_fraction": 0.65, + "metric": 0.6141666666666666, + "rescued": 3, + "harmed": 453, + "missed_rescue": 0, + "avoided_harm": 404, + "selected_both_correct": 314, + "selected_both_wrong": 10 + }, + { + "budget": 0.66, + "threshold": -20.909618377685547, + "operator": "ge", + "actual_fraction": 0.66, + "metric": 0.6041666666666666, + "rescued": 3, + "harmed": 465, + "missed_rescue": 0, + "avoided_harm": 392, + "selected_both_correct": 314, + "selected_both_wrong": 10 + }, + { + "budget": 0.67, + "threshold": -21.176576614379883, + "operator": "ge", + "actual_fraction": 0.67, + "metric": 0.5941666666666666, + "rescued": 3, + "harmed": 477, + "missed_rescue": 0, + "avoided_harm": 380, + "selected_both_correct": 314, + "selected_both_wrong": 10 + }, + { + "budget": 0.68, + "threshold": -21.371265411376953, + "operator": "ge", + "actual_fraction": 0.68, + "metric": 0.585, + "rescued": 3, + "harmed": 488, + "missed_rescue": 0, + "avoided_harm": 369, + "selected_both_correct": 315, + "selected_both_wrong": 10 + }, + { + "budget": 0.69, + "threshold": -21.526351928710938, + "operator": "ge", + "actual_fraction": 0.6891666666666667, + "metric": 0.5758333333333333, + "rescued": 3, + "harmed": 499, + "missed_rescue": 0, + "avoided_harm": 358, + "selected_both_correct": 315, + "selected_both_wrong": 10 + }, + { + "budget": 0.7, + "threshold": -21.907325744628906, + "operator": "ge", + "actual_fraction": 0.7, + "metric": 0.5675, + "rescued": 3, + "harmed": 509, + "missed_rescue": 0, + "avoided_harm": 348, + "selected_both_correct": 318, + "selected_both_wrong": 10 + }, + { + "budget": 0.71, + "threshold": -22.099746704101562, + "operator": "ge", + "actual_fraction": 0.71, + "metric": 0.5575, + "rescued": 3, + "harmed": 521, + "missed_rescue": 0, + "avoided_harm": 336, + "selected_both_correct": 318, + "selected_both_wrong": 10 + }, + { + "budget": 0.72, + "threshold": -22.31593894958496, + "operator": "ge", + "actual_fraction": 0.72, + "metric": 0.5483333333333333, + "rescued": 3, + "harmed": 532, + "missed_rescue": 0, + "avoided_harm": 325, + "selected_both_correct": 319, + "selected_both_wrong": 10 + }, + { + "budget": 0.73, + "threshold": -22.608314514160156, + "operator": "ge", + "actual_fraction": 0.73, + "metric": 0.5383333333333333, + "rescued": 3, + "harmed": 544, + "missed_rescue": 0, + "avoided_harm": 313, + "selected_both_correct": 319, + "selected_both_wrong": 10 + }, + { + "budget": 0.74, + "threshold": -22.79630470275879, + "operator": "ge", + "actual_fraction": 0.74, + "metric": 0.5291666666666667, + "rescued": 3, + "harmed": 555, + "missed_rescue": 0, + "avoided_harm": 302, + "selected_both_correct": 320, + "selected_both_wrong": 10 + }, + { + "budget": 0.75, + "threshold": -23.147855758666992, + "operator": "ge", + "actual_fraction": 0.75, + "metric": 0.52, + "rescued": 3, + "harmed": 566, + "missed_rescue": 0, + "avoided_harm": 291, + "selected_both_correct": 321, + "selected_both_wrong": 10 + }, + { + "budget": 0.76, + "threshold": -23.486713409423828, + "operator": "ge", + "actual_fraction": 0.76, + "metric": 0.51, + "rescued": 3, + "harmed": 578, + "missed_rescue": 0, + "avoided_harm": 279, + "selected_both_correct": 321, + "selected_both_wrong": 10 + }, + { + "budget": 0.77, + "threshold": -23.908952713012695, + "operator": "ge", + "actual_fraction": 0.77, + "metric": 0.5008333333333334, + "rescued": 3, + "harmed": 589, + "missed_rescue": 0, + "avoided_harm": 268, + "selected_both_correct": 322, + "selected_both_wrong": 10 + }, + { + "budget": 0.78, + "threshold": -24.201642990112305, + "operator": "ge", + "actual_fraction": 0.78, + "metric": 0.49083333333333334, + "rescued": 3, + "harmed": 601, + "missed_rescue": 0, + "avoided_harm": 256, + "selected_both_correct": 322, + "selected_both_wrong": 10 + }, + { + "budget": 0.79, + "threshold": -24.47976303100586, + "operator": "ge", + "actual_fraction": 0.79, + "metric": 0.48083333333333333, + "rescued": 3, + "harmed": 613, + "missed_rescue": 0, + "avoided_harm": 244, + "selected_both_correct": 322, + "selected_both_wrong": 10 + }, + { + "budget": 0.8, + "threshold": -24.668521881103516, + "operator": "ge", + "actual_fraction": 0.8, + "metric": 0.4708333333333333, + "rescued": 3, + "harmed": 625, + "missed_rescue": 0, + "avoided_harm": 232, + "selected_both_correct": 322, + "selected_both_wrong": 10 + }, + { + "budget": 0.81, + "threshold": -25.003650665283203, + "operator": "ge", + "actual_fraction": 0.81, + "metric": 0.4625, + "rescued": 3, + "harmed": 635, + "missed_rescue": 0, + "avoided_harm": 222, + "selected_both_correct": 324, + "selected_both_wrong": 10 + }, + { + "budget": 0.82, + "threshold": -25.187009811401367, + "operator": "ge", + "actual_fraction": 0.8191666666666667, + "metric": 0.4533333333333333, + "rescued": 3, + "harmed": 646, + "missed_rescue": 0, + "avoided_harm": 211, + "selected_both_correct": 324, + "selected_both_wrong": 10 + }, + { + "budget": 0.83, + "threshold": -25.763282775878906, + "operator": "ge", + "actual_fraction": 0.83, + "metric": 0.44416666666666665, + "rescued": 3, + "harmed": 657, + "missed_rescue": 0, + "avoided_harm": 200, + "selected_both_correct": 326, + "selected_both_wrong": 10 + }, + { + "budget": 0.84, + "threshold": -26.008209228515625, + "operator": "ge", + "actual_fraction": 0.84, + "metric": 0.43416666666666665, + "rescued": 3, + "harmed": 669, + "missed_rescue": 0, + "avoided_harm": 188, + "selected_both_correct": 326, + "selected_both_wrong": 10 + }, + { + "budget": 0.85, + "threshold": -26.451644897460938, + "operator": "ge", + "actual_fraction": 0.85, + "metric": 0.4241666666666667, + "rescued": 3, + "harmed": 681, + "missed_rescue": 0, + "avoided_harm": 176, + "selected_both_correct": 326, + "selected_both_wrong": 10 + }, + { + "budget": 0.86, + "threshold": -26.866626739501953, + "operator": "ge", + "actual_fraction": 0.86, + "metric": 0.415, + "rescued": 3, + "harmed": 692, + "missed_rescue": 0, + "avoided_harm": 165, + "selected_both_correct": 327, + "selected_both_wrong": 10 + }, + { + "budget": 0.87, + "threshold": -27.2409725189209, + "operator": "ge", + "actual_fraction": 0.87, + "metric": 0.405, + "rescued": 3, + "harmed": 704, + "missed_rescue": 0, + "avoided_harm": 153, + "selected_both_correct": 327, + "selected_both_wrong": 10 + }, + { + "budget": 0.88, + "threshold": -27.388290405273438, + "operator": "ge", + "actual_fraction": 0.88, + "metric": 0.395, + "rescued": 3, + "harmed": 716, + "missed_rescue": 0, + "avoided_harm": 141, + "selected_both_correct": 327, + "selected_both_wrong": 10 + }, + { + "budget": 0.89, + "threshold": -27.89401626586914, + "operator": "ge", + "actual_fraction": 0.89, + "metric": 0.385, + "rescued": 3, + "harmed": 728, + "missed_rescue": 0, + "avoided_harm": 129, + "selected_both_correct": 327, + "selected_both_wrong": 10 + }, + { + "budget": 0.9, + "threshold": -28.421382904052734, + "operator": "ge", + "actual_fraction": 0.9, + "metric": 0.375, + "rescued": 3, + "harmed": 740, + "missed_rescue": 0, + "avoided_harm": 117, + "selected_both_correct": 327, + "selected_both_wrong": 10 + }, + { + "budget": 0.91, + "threshold": -28.676851272583008, + "operator": "ge", + "actual_fraction": 0.91, + "metric": 0.365, + "rescued": 3, + "harmed": 752, + "missed_rescue": 0, + "avoided_harm": 105, + "selected_both_correct": 327, + "selected_both_wrong": 10 + }, + { + "budget": 0.92, + "threshold": -29.461149215698242, + "operator": "ge", + "actual_fraction": 0.92, + "metric": 0.355, + "rescued": 3, + "harmed": 764, + "missed_rescue": 0, + "avoided_harm": 93, + "selected_both_correct": 327, + "selected_both_wrong": 10 + }, + { + "budget": 0.93, + "threshold": -30.00433349609375, + "operator": "ge", + "actual_fraction": 0.93, + "metric": 0.3458333333333333, + "rescued": 3, + "harmed": 775, + "missed_rescue": 0, + "avoided_harm": 82, + "selected_both_correct": 328, + "selected_both_wrong": 10 + }, + { + "budget": 0.94, + "threshold": -30.70661163330078, + "operator": "ge", + "actual_fraction": 0.94, + "metric": 0.33666666666666667, + "rescued": 3, + "harmed": 786, + "missed_rescue": 0, + "avoided_harm": 71, + "selected_both_correct": 329, + "selected_both_wrong": 10 + }, + { + "budget": 0.95, + "threshold": -31.270946502685547, + "operator": "ge", + "actual_fraction": 0.95, + "metric": 0.3275, + "rescued": 3, + "harmed": 797, + "missed_rescue": 0, + "avoided_harm": 60, + "selected_both_correct": 330, + "selected_both_wrong": 10 + }, + { + "budget": 0.96, + "threshold": -31.816049575805664, + "operator": "ge", + "actual_fraction": 0.96, + "metric": 0.3175, + "rescued": 3, + "harmed": 809, + "missed_rescue": 0, + "avoided_harm": 48, + "selected_both_correct": 330, + "selected_both_wrong": 10 + }, + { + "budget": 0.97, + "threshold": -32.541770935058594, + "operator": "ge", + "actual_fraction": 0.97, + "metric": 0.3075, + "rescued": 3, + "harmed": 821, + "missed_rescue": 0, + "avoided_harm": 36, + "selected_both_correct": 330, + "selected_both_wrong": 10 + }, + { + "budget": 0.98, + "threshold": -33.29741287231445, + "operator": "ge", + "actual_fraction": 0.98, + "metric": 0.2975, + "rescued": 3, + "harmed": 833, + "missed_rescue": 0, + "avoided_harm": 24, + "selected_both_correct": 330, + "selected_both_wrong": 10 + }, + { + "budget": 0.99, + "threshold": -35.03510665893555, + "operator": "ge", + "actual_fraction": 0.99, + "metric": 0.2875, + "rescued": 3, + "harmed": 845, + "missed_rescue": 0, + "avoided_harm": 12, + "selected_both_correct": 330, + "selected_both_wrong": 10 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.2775, + "rescued": 3, + "harmed": 857, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 330, + "selected_both_wrong": 10 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9016223679668622, + "rescued": 0, + "harmed": 0, + "missed_rescue": 106, + "avoided_harm": 3791, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 0.5851550102233887, + "operator": "ge", + "actual_fraction": 0.022954780807732136, + "metric": 0.8997238522609596, + "rescued": 5, + "harmed": 16, + "missed_rescue": 101, + "avoided_harm": 3775, + "selected_both_correct": 87, + "selected_both_wrong": 25 + }, + { + "budget": 0.02, + "threshold": -0.7202239036560059, + "operator": "ge", + "actual_fraction": 0.03451846738004832, + "metric": 0.8988608905764585, + "rescued": 6, + "harmed": 22, + "missed_rescue": 100, + "avoided_harm": 3769, + "selected_both_correct": 133, + "selected_both_wrong": 39 + }, + { + "budget": 0.03, + "threshold": -1.5600132942199707, + "operator": "ge", + "actual_fraction": 0.04487400759406282, + "metric": 0.8979979288919572, + "rescued": 9, + "harmed": 30, + "missed_rescue": 97, + "avoided_harm": 3761, + "selected_both_correct": 171, + "selected_both_wrong": 50 + }, + { + "budget": 0.04, + "threshold": -2.136852741241455, + "operator": "ge", + "actual_fraction": 0.054366586123576116, + "metric": 0.8962720055229548, + "rescued": 9, + "harmed": 40, + "missed_rescue": 97, + "avoided_harm": 3751, + "selected_both_correct": 210, + "selected_both_wrong": 56 + }, + { + "budget": 0.05, + "threshold": -3.080942392349243, + "operator": "gt", + "actual_fraction": 0.06834656541249569, + "metric": 0.8952364515015533, + "rescued": 11, + "harmed": 48, + "missed_rescue": 95, + "avoided_harm": 3743, + "selected_both_correct": 277, + "selected_both_wrong": 60 + }, + { + "budget": 0.06, + "threshold": -3.58967924118042, + "operator": "ge", + "actual_fraction": 0.07921988263721091, + "metric": 0.8938557128063513, + "rescued": 13, + "harmed": 58, + "missed_rescue": 93, + "avoided_harm": 3733, + "selected_both_correct": 320, + "selected_both_wrong": 68 + }, + { + "budget": 0.07, + "threshold": -4.303406715393066, + "operator": "ge", + "actual_fraction": 0.09389023127373144, + "metric": 0.8917846047635485, + "rescued": 15, + "harmed": 72, + "missed_rescue": 91, + "avoided_harm": 3719, + "selected_both_correct": 377, + "selected_both_wrong": 80 + }, + { + "budget": 0.08, + "threshold": -4.7087507247924805, + "operator": "ge", + "actual_fraction": 0.10372799447704521, + "metric": 0.8898860890576459, + "rescued": 16, + "harmed": 84, + "missed_rescue": 90, + "avoided_harm": 3707, + "selected_both_correct": 417, + "selected_both_wrong": 84 + }, + { + "budget": 0.09, + "threshold": -5.179437637329102, + "operator": "ge", + "actual_fraction": 0.11408353469105972, + "metric": 0.8879875733517432, + "rescued": 18, + "harmed": 97, + "missed_rescue": 88, + "avoided_harm": 3694, + "selected_both_correct": 451, + "selected_both_wrong": 95 + }, + { + "budget": 0.1, + "threshold": -5.7383904457092285, + "operator": "ge", + "actual_fraction": 0.12858129099068002, + "metric": 0.8853986882982395, + "rescued": 21, + "harmed": 115, + "missed_rescue": 85, + "avoided_harm": 3676, + "selected_both_correct": 508, + "selected_both_wrong": 101 + }, + { + "budget": 0.11, + "threshold": -6.130518913269043, + "operator": "ge", + "actual_fraction": 0.13979979288919572, + "metric": 0.8822920262340352, + "rescued": 25, + "harmed": 137, + "missed_rescue": 81, + "avoided_harm": 3654, + "selected_both_correct": 541, + "selected_both_wrong": 107 + }, + { + "budget": 0.12, + "threshold": -6.5530104637146, + "operator": "ge", + "actual_fraction": 0.14981014842940973, + "metric": 0.8793579565067311, + "rescued": 26, + "harmed": 155, + "missed_rescue": 80, + "avoided_harm": 3636, + "selected_both_correct": 573, + "selected_both_wrong": 114 + }, + { + "budget": 0.13, + "threshold": -6.861036777496338, + "operator": "ge", + "actual_fraction": 0.15688643424231963, + "metric": 0.8774594408008285, + "rescued": 26, + "harmed": 166, + "missed_rescue": 80, + "avoided_harm": 3625, + "selected_both_correct": 597, + "selected_both_wrong": 120 + }, + { + "budget": 0.14, + "threshold": -7.096773147583008, + "operator": "ge", + "actual_fraction": 0.1629271660338281, + "metric": 0.8765964791163272, + "rescued": 27, + "harmed": 172, + "missed_rescue": 79, + "avoided_harm": 3619, + "selected_both_correct": 621, + "selected_both_wrong": 124 + }, + { + "budget": 0.15, + "threshold": -7.499796390533447, + "operator": "ge", + "actual_fraction": 0.17449085260614428, + "metric": 0.8731446323783224, + "rescued": 27, + "harmed": 192, + "missed_rescue": 79, + "avoided_harm": 3599, + "selected_both_correct": 661, + "selected_both_wrong": 131 + }, + { + "budget": 0.16, + "threshold": -7.857483386993408, + "operator": "ge", + "actual_fraction": 0.1860545391784605, + "metric": 0.8690024162927166, + "rescued": 29, + "harmed": 218, + "missed_rescue": 77, + "avoided_harm": 3573, + "selected_both_correct": 693, + "selected_both_wrong": 138 + }, + { + "budget": 0.17, + "threshold": -8.096627235412598, + "operator": "ge", + "actual_fraction": 0.19433897134967207, + "metric": 0.8660683465654125, + "rescued": 29, + "harmed": 235, + "missed_rescue": 77, + "avoided_harm": 3556, + "selected_both_correct": 723, + "selected_both_wrong": 139 + }, + { + "budget": 0.18, + "threshold": -8.421232223510742, + "operator": "ge", + "actual_fraction": 0.20590265792198825, + "metric": 0.8617535381429065, + "rescued": 31, + "harmed": 262, + "missed_rescue": 75, + "avoided_harm": 3529, + "selected_both_correct": 755, + "selected_both_wrong": 145 + }, + { + "budget": 0.19, + "threshold": -8.759540557861328, + "operator": "ge", + "actual_fraction": 0.21505005177770106, + "metric": 0.8589920607525026, + "rescued": 35, + "harmed": 282, + "missed_rescue": 71, + "avoided_harm": 3509, + "selected_both_correct": 779, + "selected_both_wrong": 150 + }, + { + "budget": 0.2, + "threshold": -9.157719612121582, + "operator": "ge", + "actual_fraction": 0.23041076976182256, + "metric": 0.8538142906454953, + "rescued": 36, + "harmed": 313, + "missed_rescue": 70, + "avoided_harm": 3478, + "selected_both_correct": 826, + "selected_both_wrong": 160 + }, + { + "budget": 0.21, + "threshold": -9.493932723999023, + "operator": "ge", + "actual_fraction": 0.2412840869865378, + "metric": 0.85001725923369, + "rescued": 39, + "harmed": 338, + "missed_rescue": 67, + "avoided_harm": 3453, + "selected_both_correct": 857, + "selected_both_wrong": 164 + }, + { + "budget": 0.22, + "threshold": -9.7905912399292, + "operator": "ge", + "actual_fraction": 0.2502588885053504, + "metric": 0.847083189506386, + "rescued": 41, + "harmed": 357, + "missed_rescue": 65, + "avoided_harm": 3434, + "selected_both_correct": 881, + "selected_both_wrong": 171 + }, + { + "budget": 0.23, + "threshold": -10.482229232788086, + "operator": "ge", + "actual_fraction": 0.2747670003451847, + "metric": 0.8369002416292717, + "rescued": 46, + "harmed": 421, + "missed_rescue": 60, + "avoided_harm": 3370, + "selected_both_correct": 938, + "selected_both_wrong": 187 + }, + { + "budget": 0.24, + "threshold": -10.810736656188965, + "operator": "ge", + "actual_fraction": 0.29081808767690714, + "metric": 0.829996548153262, + "rescued": 48, + "harmed": 463, + "missed_rescue": 58, + "avoided_harm": 3328, + "selected_both_correct": 980, + "selected_both_wrong": 194 + }, + { + "budget": 0.25, + "threshold": -11.235258102416992, + "operator": "ge", + "actual_fraction": 0.3046254746289265, + "metric": 0.8234380393510529, + "rescued": 49, + "harmed": 502, + "missed_rescue": 57, + "avoided_harm": 3289, + "selected_both_correct": 1008, + "selected_both_wrong": 206 + }, + { + "budget": 0.26, + "threshold": -11.582498550415039, + "operator": "ge", + "actual_fraction": 0.31774249223334483, + "metric": 0.8175698998964446, + "rescued": 51, + "harmed": 538, + "missed_rescue": 55, + "avoided_harm": 3253, + "selected_both_correct": 1036, + "selected_both_wrong": 216 + }, + { + "budget": 0.27, + "threshold": -11.94222640991211, + "operator": "ge", + "actual_fraction": 0.33310321021746636, + "metric": 0.8103210217466345, + "rescued": 54, + "harmed": 583, + "missed_rescue": 52, + "avoided_harm": 3208, + "selected_both_correct": 1068, + "selected_both_wrong": 225 + }, + { + "budget": 0.28, + "threshold": -12.258855819702148, + "operator": "ge", + "actual_fraction": 0.3470831895063859, + "metric": 0.8018639972385226, + "rescued": 54, + "harmed": 632, + "missed_rescue": 52, + "avoided_harm": 3159, + "selected_both_correct": 1090, + "selected_both_wrong": 235 + }, + { + "budget": 0.29, + "threshold": -12.57064437866211, + "operator": "ge", + "actual_fraction": 0.35795650673110113, + "metric": 0.795650673110114, + "rescued": 55, + "harmed": 669, + "missed_rescue": 51, + "avoided_harm": 3122, + "selected_both_correct": 1108, + "selected_both_wrong": 242 + }, + { + "budget": 0.3, + "threshold": -12.848037719726562, + "operator": "ge", + "actual_fraction": 0.372109078356921, + "metric": 0.7870210562651019, + "rescued": 56, + "harmed": 720, + "missed_rescue": 50, + "avoided_harm": 3071, + "selected_both_correct": 1130, + "selected_both_wrong": 250 + }, + { + "budget": 0.31, + "threshold": -12.99221420288086, + "operator": "ge", + "actual_fraction": 0.3781498101484294, + "metric": 0.7830514325163963, + "rescued": 56, + "harmed": 743, + "missed_rescue": 50, + "avoided_harm": 3048, + "selected_both_correct": 1139, + "selected_both_wrong": 253 + }, + { + "budget": 0.32, + "threshold": -13.233619689941406, + "operator": "ge", + "actual_fraction": 0.3855712806351398, + "metric": 0.77856403175699, + "rescued": 56, + "harmed": 769, + "missed_rescue": 50, + "avoided_harm": 3022, + "selected_both_correct": 1153, + "selected_both_wrong": 256 + }, + { + "budget": 0.33, + "threshold": -13.389532089233398, + "operator": "ge", + "actual_fraction": 0.392129789437349, + "metric": 0.7754573696927857, + "rescued": 57, + "harmed": 788, + "missed_rescue": 49, + "avoided_harm": 3003, + "selected_both_correct": 1170, + "selected_both_wrong": 257 + }, + { + "budget": 0.34, + "threshold": -13.660714149475098, + "operator": "ge", + "actual_fraction": 0.402140144977563, + "metric": 0.7690714532274767, + "rescued": 57, + "harmed": 825, + "missed_rescue": 49, + "avoided_harm": 2966, + "selected_both_correct": 1187, + "selected_both_wrong": 261 + }, + { + "budget": 0.35, + "threshold": -13.987701416015625, + "operator": "ge", + "actual_fraction": 0.4166379012771833, + "metric": 0.7613047980669658, + "rescued": 61, + "harmed": 874, + "missed_rescue": 45, + "avoided_harm": 2917, + "selected_both_correct": 1208, + "selected_both_wrong": 271 + }, + { + "budget": 0.36, + "threshold": -14.314142227172852, + "operator": "ge", + "actual_fraction": 0.4301001035554021, + "metric": 0.7519848118743527, + "rescued": 63, + "harmed": 930, + "missed_rescue": 43, + "avoided_harm": 2861, + "selected_both_correct": 1222, + "selected_both_wrong": 277 + }, + { + "budget": 0.37, + "threshold": -14.651002883911133, + "operator": "ge", + "actual_fraction": 0.4440800828443217, + "metric": 0.7409389023127373, + "rescued": 64, + "harmed": 995, + "missed_rescue": 42, + "avoided_harm": 2796, + "selected_both_correct": 1235, + "selected_both_wrong": 279 + }, + { + "budget": 0.38, + "threshold": -14.930255889892578, + "operator": "ge", + "actual_fraction": 0.45667932343803935, + "metric": 0.7317915084570245, + "rescued": 64, + "harmed": 1048, + "missed_rescue": 42, + "avoided_harm": 2743, + "selected_both_correct": 1247, + "selected_both_wrong": 287 + }, + { + "budget": 0.39, + "threshold": -15.176605224609375, + "operator": "ge", + "actual_fraction": 0.4677252329996548, + "metric": 0.7231618916120124, + "rescued": 65, + "harmed": 1099, + "missed_rescue": 41, + "avoided_harm": 2692, + "selected_both_correct": 1254, + "selected_both_wrong": 292 + }, + { + "budget": 0.4, + "threshold": -15.31328010559082, + "operator": "ge", + "actual_fraction": 0.47411114946496374, + "metric": 0.7179841215050051, + "rescued": 65, + "harmed": 1129, + "missed_rescue": 41, + "avoided_harm": 2662, + "selected_both_correct": 1261, + "selected_both_wrong": 292 + }, + { + "budget": 0.41, + "threshold": -15.492185592651367, + "operator": "ge", + "actual_fraction": 0.4820503969623749, + "metric": 0.711425612702796, + "rescued": 65, + "harmed": 1167, + "missed_rescue": 41, + "avoided_harm": 2624, + "selected_both_correct": 1265, + "selected_both_wrong": 296 + }, + { + "budget": 0.42, + "threshold": -15.74117374420166, + "operator": "ge", + "actual_fraction": 0.4917155678287884, + "metric": 0.7038315498791854, + "rescued": 65, + "harmed": 1211, + "missed_rescue": 41, + "avoided_harm": 2580, + "selected_both_correct": 1272, + "selected_both_wrong": 301 + }, + { + "budget": 0.43, + "threshold": -16.078542709350586, + "operator": "ge", + "actual_fraction": 0.5055229547808078, + "metric": 0.6939937866758716, + "rescued": 69, + "harmed": 1272, + "missed_rescue": 37, + "avoided_harm": 2519, + "selected_both_correct": 1284, + "selected_both_wrong": 304 + }, + { + "budget": 0.44, + "threshold": -16.330650329589844, + "operator": "ge", + "actual_fraction": 0.5144977562996202, + "metric": 0.6865723161891611, + "rescued": 71, + "harmed": 1317, + "missed_rescue": 35, + "avoided_harm": 2474, + "selected_both_correct": 1286, + "selected_both_wrong": 307 + }, + { + "budget": 0.45, + "threshold": -16.650772094726562, + "operator": "ge", + "actual_fraction": 0.531411805315844, + "metric": 0.6719019675526406, + "rescued": 73, + "harmed": 1404, + "missed_rescue": 33, + "avoided_harm": 2387, + "selected_both_correct": 1291, + "selected_both_wrong": 311 + }, + { + "budget": 0.46, + "threshold": -16.817930221557617, + "operator": "ge", + "actual_fraction": 0.5371073524335519, + "metric": 0.6675871591301347, + "rescued": 73, + "harmed": 1429, + "missed_rescue": 33, + "avoided_harm": 2362, + "selected_both_correct": 1293, + "selected_both_wrong": 317 + }, + { + "budget": 0.47, + "threshold": -17.027217864990234, + "operator": "ge", + "actual_fraction": 0.5443562305833621, + "metric": 0.6620642043493269, + "rescued": 73, + "harmed": 1461, + "missed_rescue": 33, + "avoided_harm": 2330, + "selected_both_correct": 1298, + "selected_both_wrong": 322 + }, + { + "budget": 0.48, + "threshold": -17.246978759765625, + "operator": "ge", + "actual_fraction": 0.5533310321021747, + "metric": 0.6546427338626165, + "rescued": 73, + "harmed": 1504, + "missed_rescue": 33, + "avoided_harm": 2287, + "selected_both_correct": 1300, + "selected_both_wrong": 329 + }, + { + "budget": 0.49, + "threshold": -17.445709228515625, + "operator": "ge", + "actual_fraction": 0.5619606489471868, + "metric": 0.6472212633759061, + "rescued": 73, + "harmed": 1547, + "missed_rescue": 33, + "avoided_harm": 2244, + "selected_both_correct": 1303, + "selected_both_wrong": 333 + }, + { + "budget": 0.5, + "threshold": -17.677627563476562, + "operator": "ge", + "actual_fraction": 0.5726613738350017, + "metric": 0.6384190541939938, + "rescued": 74, + "harmed": 1599, + "missed_rescue": 32, + "avoided_harm": 2192, + "selected_both_correct": 1308, + "selected_both_wrong": 337 + }, + { + "budget": 0.51, + "threshold": -17.777769088745117, + "operator": "ge", + "actual_fraction": 0.5766309975837073, + "metric": 0.6344494304452882, + "rescued": 74, + "harmed": 1622, + "missed_rescue": 32, + "avoided_harm": 2169, + "selected_both_correct": 1308, + "selected_both_wrong": 337 + }, + { + "budget": 0.52, + "threshold": -17.976903915405273, + "operator": "ge", + "actual_fraction": 0.586468760787021, + "metric": 0.6261649982740767, + "rescued": 75, + "harmed": 1671, + "missed_rescue": 31, + "avoided_harm": 2120, + "selected_both_correct": 1313, + "selected_both_wrong": 339 + }, + { + "budget": 0.53, + "threshold": -18.135372161865234, + "operator": "ge", + "actual_fraction": 0.593545046599931, + "metric": 0.6197790818087677, + "rescued": 75, + "harmed": 1708, + "missed_rescue": 31, + "avoided_harm": 2083, + "selected_both_correct": 1317, + "selected_both_wrong": 339 + }, + { + "budget": 0.54, + "threshold": -18.381885528564453, + "operator": "ge", + "actual_fraction": 0.6045909561615465, + "metric": 0.6102865032792544, + "rescued": 76, + "harmed": 1764, + "missed_rescue": 30, + "avoided_harm": 2027, + "selected_both_correct": 1321, + "selected_both_wrong": 342 + }, + { + "budget": 0.55, + "threshold": -18.523502349853516, + "operator": "ge", + "actual_fraction": 0.6095961339316535, + "metric": 0.6071798412150501, + "rescued": 76, + "harmed": 1782, + "missed_rescue": 30, + "avoided_harm": 2009, + "selected_both_correct": 1329, + "selected_both_wrong": 345 + }, + { + "budget": 0.56, + "threshold": -18.71038246154785, + "operator": "ge", + "actual_fraction": 0.6194338971349672, + "metric": 0.5988954090438384, + "rescued": 77, + "harmed": 1831, + "missed_rescue": 29, + "avoided_harm": 1960, + "selected_both_correct": 1334, + "selected_both_wrong": 347 + }, + { + "budget": 0.57, + "threshold": -18.846454620361328, + "operator": "ge", + "actual_fraction": 0.6247842595788747, + "metric": 0.5954435623058336, + "rescued": 79, + "harmed": 1853, + "missed_rescue": 27, + "avoided_harm": 1938, + "selected_both_correct": 1337, + "selected_both_wrong": 351 + }, + { + "budget": 0.58, + "threshold": -19.10778045654297, + "operator": "ge", + "actual_fraction": 0.6368657231618916, + "metric": 0.586468760787021, + "rescued": 81, + "harmed": 1907, + "missed_rescue": 25, + "avoided_harm": 1884, + "selected_both_correct": 1345, + "selected_both_wrong": 357 + }, + { + "budget": 0.59, + "threshold": -19.334182739257812, + "operator": "ge", + "actual_fraction": 0.6444597859855022, + "metric": 0.5799102519848118, + "rescued": 81, + "harmed": 1945, + "missed_rescue": 25, + "avoided_harm": 1846, + "selected_both_correct": 1347, + "selected_both_wrong": 361 + }, + { + "budget": 0.6, + "threshold": -19.49287986755371, + "operator": "ge", + "actual_fraction": 0.64998274076631, + "metric": 0.5749050742147048, + "rescued": 81, + "harmed": 1974, + "missed_rescue": 25, + "avoided_harm": 1817, + "selected_both_correct": 1348, + "selected_both_wrong": 363 + }, + { + "budget": 0.61, + "threshold": -19.699918746948242, + "operator": "ge", + "actual_fraction": 0.6587849499482223, + "metric": 0.5676561960648947, + "rescued": 82, + "harmed": 2017, + "missed_rescue": 24, + "avoided_harm": 1774, + "selected_both_correct": 1351, + "selected_both_wrong": 367 + }, + { + "budget": 0.62, + "threshold": -19.920011520385742, + "operator": "ge", + "actual_fraction": 0.6675871591301347, + "metric": 0.5602347255781843, + "rescued": 83, + "harmed": 2061, + "missed_rescue": 23, + "avoided_harm": 1730, + "selected_both_correct": 1355, + "selected_both_wrong": 369 + }, + { + "budget": 0.63, + "threshold": -20.140621185302734, + "operator": "ge", + "actual_fraction": 0.6769071453227476, + "metric": 0.5524680704176734, + "rescued": 85, + "harmed": 2108, + "missed_rescue": 21, + "avoided_harm": 1683, + "selected_both_correct": 1359, + "selected_both_wrong": 370 + }, + { + "budget": 0.64, + "threshold": -20.324838638305664, + "operator": "ge", + "actual_fraction": 0.6841560234725578, + "metric": 0.5471177079737659, + "rescued": 85, + "harmed": 2139, + "missed_rescue": 21, + "avoided_harm": 1652, + "selected_both_correct": 1364, + "selected_both_wrong": 376 + }, + { + "budget": 0.65, + "threshold": -20.663583755493164, + "operator": "ge", + "actual_fraction": 0.6989989644459786, + "metric": 0.5345184673800484, + "rescued": 86, + "harmed": 2213, + "missed_rescue": 20, + "avoided_harm": 1578, + "selected_both_correct": 1370, + "selected_both_wrong": 381 + }, + { + "budget": 0.66, + "threshold": -20.909618377685547, + "operator": "ge", + "actual_fraction": 0.7090093199861927, + "metric": 0.5262340352088367, + "rescued": 89, + "harmed": 2264, + "missed_rescue": 17, + "avoided_harm": 1527, + "selected_both_correct": 1370, + "selected_both_wrong": 385 + }, + { + "budget": 0.67, + "threshold": -21.176576614379883, + "operator": "ge", + "actual_fraction": 0.7188470831895064, + "metric": 0.5186399723852261, + "rescued": 91, + "harmed": 2310, + "missed_rescue": 15, + "avoided_harm": 1481, + "selected_both_correct": 1375, + "selected_both_wrong": 389 + }, + { + "budget": 0.68, + "threshold": -21.371265411376953, + "operator": "ge", + "actual_fraction": 0.7245426303072143, + "metric": 0.5132896099413186, + "rescued": 91, + "harmed": 2341, + "missed_rescue": 15, + "avoided_harm": 1450, + "selected_both_correct": 1375, + "selected_both_wrong": 391 + }, + { + "budget": 0.69, + "threshold": -21.526351928710938, + "operator": "ge", + "actual_fraction": 0.7292026234035209, + "metric": 0.5091473938557128, + "rescued": 91, + "harmed": 2365, + "missed_rescue": 15, + "avoided_harm": 1426, + "selected_both_correct": 1375, + "selected_both_wrong": 394 + }, + { + "budget": 0.7, + "threshold": -21.907325744628906, + "operator": "ge", + "actual_fraction": 0.7430100103555403, + "metric": 0.49741111494649637, + "rescued": 92, + "harmed": 2434, + "missed_rescue": 14, + "avoided_harm": 1357, + "selected_both_correct": 1380, + "selected_both_wrong": 399 + }, + { + "budget": 0.71, + "threshold": -22.099746704101562, + "operator": "ge", + "actual_fraction": 0.7497411114946496, + "metric": 0.4917155678287884, + "rescued": 93, + "harmed": 2468, + "missed_rescue": 13, + "avoided_harm": 1323, + "selected_both_correct": 1382, + "selected_both_wrong": 401 + }, + { + "budget": 0.72, + "threshold": -22.31593894958496, + "operator": "ge", + "actual_fraction": 0.759233690024163, + "metric": 0.48394891266827755, + "rescued": 94, + "harmed": 2514, + "missed_rescue": 12, + "avoided_harm": 1277, + "selected_both_correct": 1386, + "selected_both_wrong": 405 + }, + { + "budget": 0.73, + "threshold": -22.608314514160156, + "operator": "ge", + "actual_fraction": 0.7702795995857784, + "metric": 0.474283741801864, + "rescued": 94, + "harmed": 2570, + "missed_rescue": 12, + "avoided_harm": 1221, + "selected_both_correct": 1391, + "selected_both_wrong": 408 + }, + { + "budget": 0.74, + "threshold": -22.79630470275879, + "operator": "ge", + "actual_fraction": 0.7780462547462893, + "metric": 0.46738004832585434, + "rescued": 94, + "harmed": 2610, + "missed_rescue": 12, + "avoided_harm": 1181, + "selected_both_correct": 1394, + "selected_both_wrong": 410 + }, + { + "budget": 0.75, + "threshold": -23.147855758666992, + "operator": "ge", + "actual_fraction": 0.7884017949603037, + "metric": 0.4580600621332413, + "rescued": 95, + "harmed": 2665, + "missed_rescue": 11, + "avoided_harm": 1126, + "selected_both_correct": 1395, + "selected_both_wrong": 413 + }, + { + "budget": 0.76, + "threshold": -23.486713409423828, + "operator": "ge", + "actual_fraction": 0.799275112185019, + "metric": 0.44787711425612703, + "rescued": 95, + "harmed": 2724, + "missed_rescue": 11, + "avoided_harm": 1067, + "selected_both_correct": 1396, + "selected_both_wrong": 416 + }, + { + "budget": 0.77, + "threshold": -23.908952713012695, + "operator": "ge", + "actual_fraction": 0.8115291681049361, + "metric": 0.4375215740421125, + "rescued": 96, + "harmed": 2785, + "missed_rescue": 10, + "avoided_harm": 1006, + "selected_both_correct": 1398, + "selected_both_wrong": 423 + }, + { + "budget": 0.78, + "threshold": -24.201642990112305, + "operator": "ge", + "actual_fraction": 0.8180876769071453, + "metric": 0.43096306523990335, + "rescued": 96, + "harmed": 2823, + "missed_rescue": 10, + "avoided_harm": 968, + "selected_both_correct": 1398, + "selected_both_wrong": 423 + }, + { + "budget": 0.79, + "threshold": -24.47976303100586, + "operator": "ge", + "actual_fraction": 0.8270624784259579, + "metric": 0.4225060407317915, + "rescued": 96, + "harmed": 2872, + "missed_rescue": 10, + "avoided_harm": 919, + "selected_both_correct": 1400, + "selected_both_wrong": 424 + }, + { + "budget": 0.8, + "threshold": -24.668521881103516, + "operator": "ge", + "actual_fraction": 0.8332758025543666, + "metric": 0.41646530894028305, + "rescued": 96, + "harmed": 2907, + "missed_rescue": 10, + "avoided_harm": 884, + "selected_both_correct": 1400, + "selected_both_wrong": 425 + }, + { + "budget": 0.81, + "threshold": -25.003650665283203, + "operator": "ge", + "actual_fraction": 0.8419054193993787, + "metric": 0.40852606144287196, + "rescued": 96, + "harmed": 2953, + "missed_rescue": 10, + "avoided_harm": 838, + "selected_both_correct": 1402, + "selected_both_wrong": 427 + }, + { + "budget": 0.82, + "threshold": -25.187009811401367, + "operator": "ge", + "actual_fraction": 0.8474283741801864, + "metric": 0.4038660683465654, + "rescued": 96, + "harmed": 2980, + "missed_rescue": 10, + "avoided_harm": 811, + "selected_both_correct": 1403, + "selected_both_wrong": 431 + }, + { + "budget": 0.83, + "threshold": -25.763282775878906, + "operator": "ge", + "actual_fraction": 0.8629616845012081, + "metric": 0.39040386606834654, + "rescued": 99, + "harmed": 3061, + "missed_rescue": 7, + "avoided_harm": 730, + "selected_both_correct": 1408, + "selected_both_wrong": 432 + }, + { + "budget": 0.84, + "threshold": -26.008209228515625, + "operator": "ge", + "actual_fraction": 0.8695201933034173, + "metric": 0.38401794960303764, + "rescued": 99, + "harmed": 3098, + "missed_rescue": 7, + "avoided_harm": 693, + "selected_both_correct": 1408, + "selected_both_wrong": 433 + }, + { + "budget": 0.85, + "threshold": -26.451644897460938, + "operator": "ge", + "actual_fraction": 0.8816016568864342, + "metric": 0.37331722471522266, + "rescued": 100, + "harmed": 3161, + "missed_rescue": 6, + "avoided_harm": 630, + "selected_both_correct": 1410, + "selected_both_wrong": 437 + }, + { + "budget": 0.86, + "threshold": -26.866626739501953, + "operator": "ge", + "actual_fraction": 0.8936831204694512, + "metric": 0.3624439074905074, + "rescued": 100, + "harmed": 3224, + "missed_rescue": 6, + "avoided_harm": 567, + "selected_both_correct": 1411, + "selected_both_wrong": 443 + }, + { + "budget": 0.87, + "threshold": -27.2409725189209, + "operator": "ge", + "actual_fraction": 0.9024853296513635, + "metric": 0.3541594753192958, + "rescued": 100, + "harmed": 3272, + "missed_rescue": 6, + "avoided_harm": 519, + "selected_both_correct": 1414, + "selected_both_wrong": 443 + }, + { + "budget": 0.88, + "threshold": -27.388290405273438, + "operator": "ge", + "actual_fraction": 0.9057645840524681, + "metric": 0.3512254055919917, + "rescued": 100, + "harmed": 3289, + "missed_rescue": 6, + "avoided_harm": 502, + "selected_both_correct": 1414, + "selected_both_wrong": 445 + }, + { + "budget": 0.89, + "threshold": -27.89401626586914, + "operator": "ge", + "actual_fraction": 0.9154297549188816, + "metric": 0.3429409734207801, + "rescued": 101, + "harmed": 3338, + "missed_rescue": 5, + "avoided_harm": 453, + "selected_both_correct": 1417, + "selected_both_wrong": 448 + }, + { + "budget": 0.9, + "threshold": -28.421382904052734, + "operator": "ge", + "actual_fraction": 0.9261304798066966, + "metric": 0.33310321021746636, + "rescued": 101, + "harmed": 3395, + "missed_rescue": 5, + "avoided_harm": 396, + "selected_both_correct": 1419, + "selected_both_wrong": 451 + }, + { + "budget": 0.91, + "threshold": -28.676851272583008, + "operator": "ge", + "actual_fraction": 0.9302726958923024, + "metric": 0.32930617880566104, + "rescued": 101, + "harmed": 3417, + "missed_rescue": 5, + "avoided_harm": 374, + "selected_both_correct": 1420, + "selected_both_wrong": 452 + }, + { + "budget": 0.92, + "threshold": -29.461149215698242, + "operator": "ge", + "actual_fraction": 0.9430445288229202, + "metric": 0.31808767690714534, + "rescued": 101, + "harmed": 3482, + "missed_rescue": 5, + "avoided_harm": 309, + "selected_both_correct": 1423, + "selected_both_wrong": 458 + }, + { + "budget": 0.93, + "threshold": -30.00433349609375, + "operator": "ge", + "actual_fraction": 0.9520193303417328, + "metric": 0.3106662064204349, + "rescued": 103, + "harmed": 3527, + "missed_rescue": 3, + "avoided_harm": 264, + "selected_both_correct": 1425, + "selected_both_wrong": 461 + }, + { + "budget": 0.94, + "threshold": -30.70661163330078, + "operator": "ge", + "actual_fraction": 0.959958577839144, + "metric": 0.30324473593372453, + "rescued": 103, + "harmed": 3570, + "missed_rescue": 3, + "avoided_harm": 221, + "selected_both_correct": 1427, + "selected_both_wrong": 462 + }, + { + "budget": 0.95, + "threshold": -31.270946502685547, + "operator": "ge", + "actual_fraction": 0.9670348636520538, + "metric": 0.2966862271315154, + "rescued": 104, + "harmed": 3609, + "missed_rescue": 2, + "avoided_harm": 182, + "selected_both_correct": 1427, + "selected_both_wrong": 463 + }, + { + "budget": 0.96, + "threshold": -31.816049575805664, + "operator": "ge", + "actual_fraction": 0.9722126337590611, + "metric": 0.29202623403520883, + "rescued": 105, + "harmed": 3637, + "missed_rescue": 1, + "avoided_harm": 154, + "selected_both_correct": 1428, + "selected_both_wrong": 463 + }, + { + "budget": 0.97, + "threshold": -32.541770935058594, + "operator": "ge", + "actual_fraction": 0.9779081808767691, + "metric": 0.28684846392820157, + "rescued": 105, + "harmed": 3667, + "missed_rescue": 1, + "avoided_harm": 124, + "selected_both_correct": 1431, + "selected_both_wrong": 463 + }, + { + "budget": 0.98, + "threshold": -33.29741287231445, + "operator": "ge", + "actual_fraction": 0.9829133586468761, + "metric": 0.2821884708318951, + "rescued": 106, + "harmed": 3695, + "missed_rescue": 0, + "avoided_harm": 96, + "selected_both_correct": 1431, + "selected_both_wrong": 463 + }, + { + "budget": 0.99, + "threshold": -35.03510665893555, + "operator": "ge", + "actual_fraction": 0.9918881601656886, + "metric": 0.27355885398688295, + "rescued": 106, + "harmed": 3745, + "missed_rescue": 0, + "avoided_harm": 46, + "selected_both_correct": 1432, + "selected_both_wrong": 464 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.26561960648947186, + "rescued": 106, + "harmed": 3791, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 1433, + "selected_both_wrong": 464 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 330, + "rescuable": 3, + "harmful": 857, + "both_wrong": 10, + "unit": "sample", + "total": 1200 + }, + "test": { + "both_correct": 1433, + "rescuable": 106, + "harmful": 3791, + "both_wrong": 464, + "unit": "sample", + "total": 5794 + } + }, + "small": { + "validation": 0.9891666666666666, + "test": 0.9016223679668622 + }, + "large": { + "validation": 0.2775, + "test": 0.26561960648947186 + }, + "learned_minus_confidence_test": 0.021056265101829452, + "learned_minus_small_test": -0.0036244390749050615, + "learned_minus_large_test": 0.6323783224024853 + }, + { + "key": [ + "cub", + "glsim", + "Qwen3.8" + ], + "expert_display": "GLSim", + "run": "shared_12_glsim_Qwen3.8", + "validation_rows": 1200, + "test_rows": 5794, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/glsim/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 1, + "validation_area_0_25": 0.9213333333333333, + "validation_point": { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9858333333333333, + "rescued": 1, + "harmed": 5, + "missed_rescue": 1, + "avoided_harm": 523, + "selected_both_correct": 1, + "selected_both_wrong": 5 + }, + "test_point": { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.08698653779772178, + "metric": 0.8878149810148429, + "rescued": 70, + "harmed": 150, + "missed_rescue": 74, + "avoided_harm": 2042, + "selected_both_correct": 85, + "selected_both_wrong": 199 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9891666666666666, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2, + "avoided_harm": 528, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9858333333333333, + "rescued": 1, + "harmed": 5, + "missed_rescue": 1, + "avoided_harm": 523, + "selected_both_correct": 1, + "selected_both_wrong": 5 + }, + { + "budget": 0.02, + "threshold": -0.6014261841773987, + "operator": "ge", + "actual_fraction": 0.02, + "metric": 0.98, + "rescued": 1, + "harmed": 12, + "missed_rescue": 1, + "avoided_harm": 516, + "selected_both_correct": 4, + "selected_both_wrong": 7 + }, + { + "budget": 0.03, + "threshold": -0.6401472687721252, + "operator": "ge", + "actual_fraction": 0.03, + "metric": 0.9741666666666666, + "rescued": 2, + "harmed": 20, + "missed_rescue": 0, + "avoided_harm": 508, + "selected_both_correct": 6, + "selected_both_wrong": 8 + }, + { + "budget": 0.04, + "threshold": -0.6884650588035583, + "operator": "ge", + "actual_fraction": 0.04, + "metric": 0.9675, + "rescued": 2, + "harmed": 28, + "missed_rescue": 0, + "avoided_harm": 500, + "selected_both_correct": 8, + "selected_both_wrong": 10 + }, + { + "budget": 0.05, + "threshold": -0.7208121418952942, + "operator": "ge", + "actual_fraction": 0.05, + "metric": 0.96, + "rescued": 2, + "harmed": 37, + "missed_rescue": 0, + "avoided_harm": 491, + "selected_both_correct": 11, + "selected_both_wrong": 10 + }, + { + "budget": 0.06, + "threshold": -0.7578181624412537, + "operator": "ge", + "actual_fraction": 0.06, + "metric": 0.9558333333333333, + "rescued": 2, + "harmed": 42, + "missed_rescue": 0, + "avoided_harm": 486, + "selected_both_correct": 18, + "selected_both_wrong": 10 + }, + { + "budget": 0.07, + "threshold": -0.7790470719337463, + "operator": "ge", + "actual_fraction": 0.07, + "metric": 0.95, + "rescued": 2, + "harmed": 49, + "missed_rescue": 0, + "avoided_harm": 479, + "selected_both_correct": 23, + "selected_both_wrong": 10 + }, + { + "budget": 0.08, + "threshold": -0.7927228808403015, + "operator": "ge", + "actual_fraction": 0.08, + "metric": 0.945, + "rescued": 2, + "harmed": 55, + "missed_rescue": 0, + "avoided_harm": 473, + "selected_both_correct": 28, + "selected_both_wrong": 11 + }, + { + "budget": 0.09, + "threshold": -0.8049252033233643, + "operator": "ge", + "actual_fraction": 0.09, + "metric": 0.9408333333333333, + "rescued": 2, + "harmed": 60, + "missed_rescue": 0, + "avoided_harm": 468, + "selected_both_correct": 35, + "selected_both_wrong": 11 + }, + { + "budget": 0.1, + "threshold": -0.80850750207901, + "operator": "ge", + "actual_fraction": 0.1, + "metric": 0.9333333333333333, + "rescued": 2, + "harmed": 69, + "missed_rescue": 0, + "avoided_harm": 459, + "selected_both_correct": 38, + "selected_both_wrong": 11 + }, + { + "budget": 0.11, + "threshold": -0.819871723651886, + "operator": "ge", + "actual_fraction": 0.11, + "metric": 0.9258333333333333, + "rescued": 2, + "harmed": 78, + "missed_rescue": 0, + "avoided_harm": 450, + "selected_both_correct": 41, + "selected_both_wrong": 11 + }, + { + "budget": 0.12, + "threshold": -0.8310170769691467, + "operator": "ge", + "actual_fraction": 0.12, + "metric": 0.9225, + "rescued": 2, + "harmed": 82, + "missed_rescue": 0, + "avoided_harm": 446, + "selected_both_correct": 49, + "selected_both_wrong": 11 + }, + { + "budget": 0.13, + "threshold": -0.8376906514167786, + "operator": "ge", + "actual_fraction": 0.13, + "metric": 0.9183333333333333, + "rescued": 2, + "harmed": 87, + "missed_rescue": 0, + "avoided_harm": 441, + "selected_both_correct": 56, + "selected_both_wrong": 11 + }, + { + "budget": 0.14, + "threshold": -0.8419997692108154, + "operator": "ge", + "actual_fraction": 0.14, + "metric": 0.9108333333333334, + "rescued": 2, + "harmed": 96, + "missed_rescue": 0, + "avoided_harm": 432, + "selected_both_correct": 59, + "selected_both_wrong": 11 + }, + { + "budget": 0.15, + "threshold": -0.8447275161743164, + "operator": "ge", + "actual_fraction": 0.15, + "metric": 0.9058333333333334, + "rescued": 2, + "harmed": 102, + "missed_rescue": 0, + "avoided_harm": 426, + "selected_both_correct": 65, + "selected_both_wrong": 11 + }, + { + "budget": 0.16, + "threshold": -0.8506776690483093, + "operator": "ge", + "actual_fraction": 0.16, + "metric": 0.9016666666666666, + "rescued": 2, + "harmed": 107, + "missed_rescue": 0, + "avoided_harm": 421, + "selected_both_correct": 72, + "selected_both_wrong": 11 + }, + { + "budget": 0.17, + "threshold": -0.8545639514923096, + "operator": "ge", + "actual_fraction": 0.17, + "metric": 0.8966666666666666, + "rescued": 2, + "harmed": 113, + "missed_rescue": 0, + "avoided_harm": 415, + "selected_both_correct": 78, + "selected_both_wrong": 11 + }, + { + "budget": 0.18, + "threshold": -0.8578683137893677, + "operator": "ge", + "actual_fraction": 0.18, + "metric": 0.8916666666666667, + "rescued": 2, + "harmed": 119, + "missed_rescue": 0, + "avoided_harm": 409, + "selected_both_correct": 84, + "selected_both_wrong": 11 + }, + { + "budget": 0.19, + "threshold": -0.86090087890625, + "operator": "ge", + "actual_fraction": 0.19, + "metric": 0.8866666666666667, + "rescued": 2, + "harmed": 125, + "missed_rescue": 0, + "avoided_harm": 403, + "selected_both_correct": 90, + "selected_both_wrong": 11 + }, + { + "budget": 0.2, + "threshold": -0.8640955090522766, + "operator": "ge", + "actual_fraction": 0.2, + "metric": 0.8841666666666667, + "rescued": 2, + "harmed": 128, + "missed_rescue": 0, + "avoided_harm": 400, + "selected_both_correct": 99, + "selected_both_wrong": 11 + }, + { + "budget": 0.21, + "threshold": -0.8664237260818481, + "operator": "ge", + "actual_fraction": 0.21, + "metric": 0.8791666666666667, + "rescued": 2, + "harmed": 134, + "missed_rescue": 0, + "avoided_harm": 394, + "selected_both_correct": 105, + "selected_both_wrong": 11 + }, + { + "budget": 0.22, + "threshold": -0.8686786890029907, + "operator": "ge", + "actual_fraction": 0.22, + "metric": 0.8741666666666666, + "rescued": 2, + "harmed": 140, + "missed_rescue": 0, + "avoided_harm": 388, + "selected_both_correct": 111, + "selected_both_wrong": 11 + }, + { + "budget": 0.23, + "threshold": -0.8709638118743896, + "operator": "ge", + "actual_fraction": 0.23, + "metric": 0.8683333333333333, + "rescued": 2, + "harmed": 147, + "missed_rescue": 0, + "avoided_harm": 381, + "selected_both_correct": 116, + "selected_both_wrong": 11 + }, + { + "budget": 0.24, + "threshold": -0.8724074959754944, + "operator": "ge", + "actual_fraction": 0.24, + "metric": 0.8625, + "rescued": 2, + "harmed": 154, + "missed_rescue": 0, + "avoided_harm": 374, + "selected_both_correct": 121, + "selected_both_wrong": 11 + }, + { + "budget": 0.25, + "threshold": -0.8736594319343567, + "operator": "ge", + "actual_fraction": 0.25, + "metric": 0.8575, + "rescued": 2, + "harmed": 160, + "missed_rescue": 0, + "avoided_harm": 368, + "selected_both_correct": 127, + "selected_both_wrong": 11 + }, + { + "budget": 0.26, + "threshold": -0.8752554059028625, + "operator": "ge", + "actual_fraction": 0.26, + "metric": 0.8533333333333334, + "rescued": 2, + "harmed": 165, + "missed_rescue": 0, + "avoided_harm": 363, + "selected_both_correct": 134, + "selected_both_wrong": 11 + }, + { + "budget": 0.27, + "threshold": -0.8766024112701416, + "operator": "ge", + "actual_fraction": 0.27, + "metric": 0.8508333333333333, + "rescued": 2, + "harmed": 168, + "missed_rescue": 0, + "avoided_harm": 360, + "selected_both_correct": 143, + "selected_both_wrong": 11 + }, + { + "budget": 0.28, + "threshold": -0.878311276435852, + "operator": "ge", + "actual_fraction": 0.28, + "metric": 0.8475, + "rescued": 2, + "harmed": 172, + "missed_rescue": 0, + "avoided_harm": 356, + "selected_both_correct": 151, + "selected_both_wrong": 11 + }, + { + "budget": 0.29, + "threshold": -0.880103588104248, + "operator": "ge", + "actual_fraction": 0.29, + "metric": 0.8425, + "rescued": 2, + "harmed": 178, + "missed_rescue": 0, + "avoided_harm": 350, + "selected_both_correct": 157, + "selected_both_wrong": 11 + }, + { + "budget": 0.3, + "threshold": -0.8824952244758606, + "operator": "ge", + "actual_fraction": 0.3, + "metric": 0.8375, + "rescued": 2, + "harmed": 184, + "missed_rescue": 0, + "avoided_harm": 344, + "selected_both_correct": 163, + "selected_both_wrong": 11 + }, + { + "budget": 0.31, + "threshold": -0.8831692337989807, + "operator": "ge", + "actual_fraction": 0.31, + "metric": 0.8358333333333333, + "rescued": 2, + "harmed": 186, + "missed_rescue": 0, + "avoided_harm": 342, + "selected_both_correct": 173, + "selected_both_wrong": 11 + }, + { + "budget": 0.32, + "threshold": -0.8845130205154419, + "operator": "ge", + "actual_fraction": 0.32, + "metric": 0.8316666666666667, + "rescued": 2, + "harmed": 191, + "missed_rescue": 0, + "avoided_harm": 337, + "selected_both_correct": 180, + "selected_both_wrong": 11 + }, + { + "budget": 0.33, + "threshold": -0.8855781555175781, + "operator": "ge", + "actual_fraction": 0.33, + "metric": 0.8291666666666667, + "rescued": 2, + "harmed": 194, + "missed_rescue": 0, + "avoided_harm": 334, + "selected_both_correct": 189, + "selected_both_wrong": 11 + }, + { + "budget": 0.34, + "threshold": -0.8865919709205627, + "operator": "ge", + "actual_fraction": 0.34, + "metric": 0.8266666666666667, + "rescued": 2, + "harmed": 197, + "missed_rescue": 0, + "avoided_harm": 331, + "selected_both_correct": 198, + "selected_both_wrong": 11 + }, + { + "budget": 0.35, + "threshold": -0.8877584338188171, + "operator": "ge", + "actual_fraction": 0.35, + "metric": 0.8208333333333333, + "rescued": 2, + "harmed": 204, + "missed_rescue": 0, + "avoided_harm": 324, + "selected_both_correct": 203, + "selected_both_wrong": 11 + }, + { + "budget": 0.36, + "threshold": -0.8890330195426941, + "operator": "ge", + "actual_fraction": 0.36, + "metric": 0.8166666666666667, + "rescued": 2, + "harmed": 209, + "missed_rescue": 0, + "avoided_harm": 319, + "selected_both_correct": 210, + "selected_both_wrong": 11 + }, + { + "budget": 0.37, + "threshold": -0.890850305557251, + "operator": "ge", + "actual_fraction": 0.37, + "metric": 0.815, + "rescued": 2, + "harmed": 211, + "missed_rescue": 0, + "avoided_harm": 317, + "selected_both_correct": 220, + "selected_both_wrong": 11 + }, + { + "budget": 0.38, + "threshold": -0.8916605710983276, + "operator": "ge", + "actual_fraction": 0.38, + "metric": 0.8125, + "rescued": 2, + "harmed": 214, + "missed_rescue": 0, + "avoided_harm": 314, + "selected_both_correct": 229, + "selected_both_wrong": 11 + }, + { + "budget": 0.39, + "threshold": -0.8928589224815369, + "operator": "ge", + "actual_fraction": 0.39, + "metric": 0.8066666666666666, + "rescued": 2, + "harmed": 221, + "missed_rescue": 0, + "avoided_harm": 307, + "selected_both_correct": 234, + "selected_both_wrong": 11 + }, + { + "budget": 0.4, + "threshold": -0.8940943479537964, + "operator": "ge", + "actual_fraction": 0.4, + "metric": 0.8, + "rescued": 2, + "harmed": 229, + "missed_rescue": 0, + "avoided_harm": 299, + "selected_both_correct": 238, + "selected_both_wrong": 11 + }, + { + "budget": 0.41, + "threshold": -0.8949463963508606, + "operator": "ge", + "actual_fraction": 0.4091666666666667, + "metric": 0.7991666666666667, + "rescued": 2, + "harmed": 230, + "missed_rescue": 0, + "avoided_harm": 298, + "selected_both_correct": 248, + "selected_both_wrong": 11 + }, + { + "budget": 0.42, + "threshold": -0.8960229158401489, + "operator": "ge", + "actual_fraction": 0.42, + "metric": 0.7941666666666667, + "rescued": 2, + "harmed": 236, + "missed_rescue": 0, + "avoided_harm": 292, + "selected_both_correct": 255, + "selected_both_wrong": 11 + }, + { + "budget": 0.43, + "threshold": -0.8966536521911621, + "operator": "ge", + "actual_fraction": 0.43, + "metric": 0.7916666666666666, + "rescued": 2, + "harmed": 239, + "missed_rescue": 0, + "avoided_harm": 289, + "selected_both_correct": 264, + "selected_both_wrong": 11 + }, + { + "budget": 0.44, + "threshold": -0.8975363373756409, + "operator": "ge", + "actual_fraction": 0.44, + "metric": 0.7891666666666667, + "rescued": 2, + "harmed": 242, + "missed_rescue": 0, + "avoided_harm": 286, + "selected_both_correct": 273, + "selected_both_wrong": 11 + }, + { + "budget": 0.45, + "threshold": -0.8988122940063477, + "operator": "ge", + "actual_fraction": 0.45, + "metric": 0.7875, + "rescued": 2, + "harmed": 244, + "missed_rescue": 0, + "avoided_harm": 284, + "selected_both_correct": 283, + "selected_both_wrong": 11 + }, + { + "budget": 0.46, + "threshold": -0.9000959396362305, + "operator": "ge", + "actual_fraction": 0.46, + "metric": 0.7816666666666666, + "rescued": 2, + "harmed": 251, + "missed_rescue": 0, + "avoided_harm": 277, + "selected_both_correct": 288, + "selected_both_wrong": 11 + }, + { + "budget": 0.47, + "threshold": -0.9013051986694336, + "operator": "ge", + "actual_fraction": 0.47, + "metric": 0.7791666666666667, + "rescued": 2, + "harmed": 254, + "missed_rescue": 0, + "avoided_harm": 274, + "selected_both_correct": 297, + "selected_both_wrong": 11 + }, + { + "budget": 0.48, + "threshold": -0.9029324054718018, + "operator": "ge", + "actual_fraction": 0.48, + "metric": 0.775, + "rescued": 2, + "harmed": 259, + "missed_rescue": 0, + "avoided_harm": 269, + "selected_both_correct": 304, + "selected_both_wrong": 11 + }, + { + "budget": 0.49, + "threshold": -0.9036871790885925, + "operator": "ge", + "actual_fraction": 0.49, + "metric": 0.7733333333333333, + "rescued": 2, + "harmed": 261, + "missed_rescue": 0, + "avoided_harm": 267, + "selected_both_correct": 314, + "selected_both_wrong": 11 + }, + { + "budget": 0.5, + "threshold": -0.9044621586799622, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.77, + "rescued": 2, + "harmed": 265, + "missed_rescue": 0, + "avoided_harm": 263, + "selected_both_correct": 322, + "selected_both_wrong": 11 + }, + { + "budget": 0.51, + "threshold": -0.9054575562477112, + "operator": "ge", + "actual_fraction": 0.51, + "metric": 0.7666666666666667, + "rescued": 2, + "harmed": 269, + "missed_rescue": 0, + "avoided_harm": 259, + "selected_both_correct": 330, + "selected_both_wrong": 11 + }, + { + "budget": 0.52, + "threshold": -0.906682550907135, + "operator": "ge", + "actual_fraction": 0.52, + "metric": 0.7616666666666667, + "rescued": 2, + "harmed": 275, + "missed_rescue": 0, + "avoided_harm": 253, + "selected_both_correct": 336, + "selected_both_wrong": 11 + }, + { + "budget": 0.53, + "threshold": -0.9072114825248718, + "operator": "ge", + "actual_fraction": 0.53, + "metric": 0.7575, + "rescued": 2, + "harmed": 280, + "missed_rescue": 0, + "avoided_harm": 248, + "selected_both_correct": 343, + "selected_both_wrong": 11 + }, + { + "budget": 0.54, + "threshold": -0.9083563089370728, + "operator": "ge", + "actual_fraction": 0.54, + "metric": 0.7533333333333333, + "rescued": 2, + "harmed": 285, + "missed_rescue": 0, + "avoided_harm": 243, + "selected_both_correct": 350, + "selected_both_wrong": 11 + }, + { + "budget": 0.55, + "threshold": -0.9095192551612854, + "operator": "ge", + "actual_fraction": 0.55, + "metric": 0.75, + "rescued": 2, + "harmed": 289, + "missed_rescue": 0, + "avoided_harm": 239, + "selected_both_correct": 358, + "selected_both_wrong": 11 + }, + { + "budget": 0.56, + "threshold": -0.9106979370117188, + "operator": "ge", + "actual_fraction": 0.56, + "metric": 0.7466666666666667, + "rescued": 2, + "harmed": 293, + "missed_rescue": 0, + "avoided_harm": 235, + "selected_both_correct": 366, + "selected_both_wrong": 11 + }, + { + "budget": 0.57, + "threshold": -0.9115362167358398, + "operator": "ge", + "actual_fraction": 0.5691666666666667, + "metric": 0.7425, + "rescued": 2, + "harmed": 298, + "missed_rescue": 0, + "avoided_harm": 230, + "selected_both_correct": 372, + "selected_both_wrong": 11 + }, + { + "budget": 0.58, + "threshold": -0.9124338030815125, + "operator": "ge", + "actual_fraction": 0.58, + "metric": 0.7375, + "rescued": 2, + "harmed": 304, + "missed_rescue": 0, + "avoided_harm": 224, + "selected_both_correct": 379, + "selected_both_wrong": 11 + }, + { + "budget": 0.59, + "threshold": -0.9134801626205444, + "operator": "ge", + "actual_fraction": 0.59, + "metric": 0.735, + "rescued": 2, + "harmed": 307, + "missed_rescue": 0, + "avoided_harm": 221, + "selected_both_correct": 388, + "selected_both_wrong": 11 + }, + { + "budget": 0.6, + "threshold": -0.9143747091293335, + "operator": "ge", + "actual_fraction": 0.6, + "metric": 0.7316666666666667, + "rescued": 2, + "harmed": 311, + "missed_rescue": 0, + "avoided_harm": 217, + "selected_both_correct": 396, + "selected_both_wrong": 11 + }, + { + "budget": 0.61, + "threshold": -0.915804386138916, + "operator": "ge", + "actual_fraction": 0.61, + "metric": 0.7283333333333334, + "rescued": 2, + "harmed": 315, + "missed_rescue": 0, + "avoided_harm": 213, + "selected_both_correct": 404, + "selected_both_wrong": 11 + }, + { + "budget": 0.62, + "threshold": -0.9165146350860596, + "operator": "ge", + "actual_fraction": 0.62, + "metric": 0.7266666666666667, + "rescued": 2, + "harmed": 317, + "missed_rescue": 0, + "avoided_harm": 211, + "selected_both_correct": 414, + "selected_both_wrong": 11 + }, + { + "budget": 0.63, + "threshold": -0.9172822833061218, + "operator": "ge", + "actual_fraction": 0.63, + "metric": 0.725, + "rescued": 2, + "harmed": 319, + "missed_rescue": 0, + "avoided_harm": 209, + "selected_both_correct": 424, + "selected_both_wrong": 11 + }, + { + "budget": 0.64, + "threshold": -0.9181071519851685, + "operator": "ge", + "actual_fraction": 0.64, + "metric": 0.7208333333333333, + "rescued": 2, + "harmed": 324, + "missed_rescue": 0, + "avoided_harm": 204, + "selected_both_correct": 431, + "selected_both_wrong": 11 + }, + { + "budget": 0.65, + "threshold": -0.9192029237747192, + "operator": "ge", + "actual_fraction": 0.65, + "metric": 0.7175, + "rescued": 2, + "harmed": 328, + "missed_rescue": 0, + "avoided_harm": 200, + "selected_both_correct": 439, + "selected_both_wrong": 11 + }, + { + "budget": 0.66, + "threshold": -0.9200543761253357, + "operator": "ge", + "actual_fraction": 0.66, + "metric": 0.7133333333333334, + "rescued": 2, + "harmed": 333, + "missed_rescue": 0, + "avoided_harm": 195, + "selected_both_correct": 446, + "selected_both_wrong": 11 + }, + { + "budget": 0.67, + "threshold": -0.9208071231842041, + "operator": "ge", + "actual_fraction": 0.67, + "metric": 0.71, + "rescued": 2, + "harmed": 337, + "missed_rescue": 0, + "avoided_harm": 191, + "selected_both_correct": 454, + "selected_both_wrong": 11 + }, + { + "budget": 0.68, + "threshold": -0.9215553998947144, + "operator": "ge", + "actual_fraction": 0.68, + "metric": 0.7058333333333333, + "rescued": 2, + "harmed": 342, + "missed_rescue": 0, + "avoided_harm": 186, + "selected_both_correct": 461, + "selected_both_wrong": 11 + }, + { + "budget": 0.69, + "threshold": -0.9220835566520691, + "operator": "ge", + "actual_fraction": 0.6891666666666667, + "metric": 0.7016666666666667, + "rescued": 2, + "harmed": 347, + "missed_rescue": 0, + "avoided_harm": 181, + "selected_both_correct": 467, + "selected_both_wrong": 11 + }, + { + "budget": 0.7, + "threshold": -0.9232237935066223, + "operator": "ge", + "actual_fraction": 0.7, + "metric": 0.6975, + "rescued": 2, + "harmed": 352, + "missed_rescue": 0, + "avoided_harm": 176, + "selected_both_correct": 475, + "selected_both_wrong": 11 + }, + { + "budget": 0.71, + "threshold": -0.9243364334106445, + "operator": "ge", + "actual_fraction": 0.71, + "metric": 0.6933333333333334, + "rescued": 2, + "harmed": 357, + "missed_rescue": 0, + "avoided_harm": 171, + "selected_both_correct": 482, + "selected_both_wrong": 11 + }, + { + "budget": 0.72, + "threshold": -0.9252541065216064, + "operator": "ge", + "actual_fraction": 0.72, + "metric": 0.69, + "rescued": 2, + "harmed": 361, + "missed_rescue": 0, + "avoided_harm": 167, + "selected_both_correct": 490, + "selected_both_wrong": 11 + }, + { + "budget": 0.73, + "threshold": -0.925781786441803, + "operator": "ge", + "actual_fraction": 0.73, + "metric": 0.6866666666666666, + "rescued": 2, + "harmed": 365, + "missed_rescue": 0, + "avoided_harm": 163, + "selected_both_correct": 498, + "selected_both_wrong": 11 + }, + { + "budget": 0.74, + "threshold": -0.9274963140487671, + "operator": "ge", + "actual_fraction": 0.74, + "metric": 0.6833333333333333, + "rescued": 2, + "harmed": 369, + "missed_rescue": 0, + "avoided_harm": 159, + "selected_both_correct": 506, + "selected_both_wrong": 11 + }, + { + "budget": 0.75, + "threshold": -0.9286423325538635, + "operator": "ge", + "actual_fraction": 0.75, + "metric": 0.6783333333333333, + "rescued": 2, + "harmed": 375, + "missed_rescue": 0, + "avoided_harm": 153, + "selected_both_correct": 512, + "selected_both_wrong": 11 + }, + { + "budget": 0.76, + "threshold": -0.9301291704177856, + "operator": "ge", + "actual_fraction": 0.76, + "metric": 0.6741666666666667, + "rescued": 2, + "harmed": 380, + "missed_rescue": 0, + "avoided_harm": 148, + "selected_both_correct": 519, + "selected_both_wrong": 11 + }, + { + "budget": 0.77, + "threshold": -0.9318777918815613, + "operator": "ge", + "actual_fraction": 0.77, + "metric": 0.67, + "rescued": 2, + "harmed": 385, + "missed_rescue": 0, + "avoided_harm": 143, + "selected_both_correct": 526, + "selected_both_wrong": 11 + }, + { + "budget": 0.78, + "threshold": -0.9332055449485779, + "operator": "ge", + "actual_fraction": 0.78, + "metric": 0.665, + "rescued": 2, + "harmed": 391, + "missed_rescue": 0, + "avoided_harm": 137, + "selected_both_correct": 532, + "selected_both_wrong": 11 + }, + { + "budget": 0.79, + "threshold": -0.9344666600227356, + "operator": "ge", + "actual_fraction": 0.79, + "metric": 0.6616666666666666, + "rescued": 2, + "harmed": 395, + "missed_rescue": 0, + "avoided_harm": 133, + "selected_both_correct": 540, + "selected_both_wrong": 11 + }, + { + "budget": 0.8, + "threshold": -0.9362465143203735, + "operator": "ge", + "actual_fraction": 0.8, + "metric": 0.6583333333333333, + "rescued": 2, + "harmed": 399, + "missed_rescue": 0, + "avoided_harm": 129, + "selected_both_correct": 548, + "selected_both_wrong": 11 + }, + { + "budget": 0.81, + "threshold": -0.937842071056366, + "operator": "ge", + "actual_fraction": 0.81, + "metric": 0.6533333333333333, + "rescued": 2, + "harmed": 405, + "missed_rescue": 0, + "avoided_harm": 123, + "selected_both_correct": 554, + "selected_both_wrong": 11 + }, + { + "budget": 0.82, + "threshold": -0.9384457468986511, + "operator": "ge", + "actual_fraction": 0.8191666666666667, + "metric": 0.65, + "rescued": 2, + "harmed": 409, + "missed_rescue": 0, + "avoided_harm": 119, + "selected_both_correct": 561, + "selected_both_wrong": 11 + }, + { + "budget": 0.83, + "threshold": -0.9396296143531799, + "operator": "ge", + "actual_fraction": 0.83, + "metric": 0.6458333333333334, + "rescued": 2, + "harmed": 414, + "missed_rescue": 0, + "avoided_harm": 114, + "selected_both_correct": 569, + "selected_both_wrong": 11 + }, + { + "budget": 0.84, + "threshold": -0.9407488703727722, + "operator": "ge", + "actual_fraction": 0.84, + "metric": 0.6383333333333333, + "rescued": 2, + "harmed": 423, + "missed_rescue": 0, + "avoided_harm": 105, + "selected_both_correct": 572, + "selected_both_wrong": 11 + }, + { + "budget": 0.85, + "threshold": -0.942738950252533, + "operator": "ge", + "actual_fraction": 0.85, + "metric": 0.6358333333333334, + "rescued": 2, + "harmed": 426, + "missed_rescue": 0, + "avoided_harm": 102, + "selected_both_correct": 581, + "selected_both_wrong": 11 + }, + { + "budget": 0.86, + "threshold": -0.9438527226448059, + "operator": "gt", + "actual_fraction": 0.8591666666666666, + "metric": 0.6325, + "rescued": 2, + "harmed": 430, + "missed_rescue": 0, + "avoided_harm": 98, + "selected_both_correct": 588, + "selected_both_wrong": 11 + }, + { + "budget": 0.87, + "threshold": -0.9450756907463074, + "operator": "ge", + "actual_fraction": 0.87, + "metric": 0.6308333333333334, + "rescued": 2, + "harmed": 432, + "missed_rescue": 0, + "avoided_harm": 96, + "selected_both_correct": 599, + "selected_both_wrong": 11 + }, + { + "budget": 0.88, + "threshold": -0.9481598138809204, + "operator": "ge", + "actual_fraction": 0.88, + "metric": 0.625, + "rescued": 2, + "harmed": 439, + "missed_rescue": 0, + "avoided_harm": 89, + "selected_both_correct": 604, + "selected_both_wrong": 11 + }, + { + "budget": 0.89, + "threshold": -0.9500836730003357, + "operator": "ge", + "actual_fraction": 0.89, + "metric": 0.62, + "rescued": 2, + "harmed": 445, + "missed_rescue": 0, + "avoided_harm": 83, + "selected_both_correct": 610, + "selected_both_wrong": 11 + }, + { + "budget": 0.9, + "threshold": -0.9519118666648865, + "operator": "ge", + "actual_fraction": 0.9, + "metric": 0.6133333333333333, + "rescued": 2, + "harmed": 453, + "missed_rescue": 0, + "avoided_harm": 75, + "selected_both_correct": 614, + "selected_both_wrong": 11 + }, + { + "budget": 0.91, + "threshold": -0.9543898701667786, + "operator": "ge", + "actual_fraction": 0.91, + "metric": 0.6066666666666667, + "rescued": 2, + "harmed": 461, + "missed_rescue": 0, + "avoided_harm": 67, + "selected_both_correct": 618, + "selected_both_wrong": 11 + }, + { + "budget": 0.92, + "threshold": -0.9570822715759277, + "operator": "ge", + "actual_fraction": 0.92, + "metric": 0.6008333333333333, + "rescued": 2, + "harmed": 468, + "missed_rescue": 0, + "avoided_harm": 60, + "selected_both_correct": 623, + "selected_both_wrong": 11 + }, + { + "budget": 0.93, + "threshold": -0.9593446254730225, + "operator": "ge", + "actual_fraction": 0.93, + "metric": 0.5941666666666666, + "rescued": 2, + "harmed": 476, + "missed_rescue": 0, + "avoided_harm": 52, + "selected_both_correct": 627, + "selected_both_wrong": 11 + }, + { + "budget": 0.94, + "threshold": -0.9610995054244995, + "operator": "ge", + "actual_fraction": 0.94, + "metric": 0.5883333333333334, + "rescued": 2, + "harmed": 483, + "missed_rescue": 0, + "avoided_harm": 45, + "selected_both_correct": 632, + "selected_both_wrong": 11 + }, + { + "budget": 0.95, + "threshold": -0.9635111093521118, + "operator": "ge", + "actual_fraction": 0.95, + "metric": 0.5825, + "rescued": 2, + "harmed": 490, + "missed_rescue": 0, + "avoided_harm": 38, + "selected_both_correct": 637, + "selected_both_wrong": 11 + }, + { + "budget": 0.96, + "threshold": -0.9648061990737915, + "operator": "ge", + "actual_fraction": 0.96, + "metric": 0.5775, + "rescued": 2, + "harmed": 496, + "missed_rescue": 0, + "avoided_harm": 32, + "selected_both_correct": 643, + "selected_both_wrong": 11 + }, + { + "budget": 0.97, + "threshold": -0.9674390554428101, + "operator": "ge", + "actual_fraction": 0.97, + "metric": 0.57, + "rescued": 2, + "harmed": 505, + "missed_rescue": 0, + "avoided_harm": 23, + "selected_both_correct": 646, + "selected_both_wrong": 11 + }, + { + "budget": 0.98, + "threshold": -0.9724470973014832, + "operator": "ge", + "actual_fraction": 0.98, + "metric": 0.5641666666666667, + "rescued": 2, + "harmed": 512, + "missed_rescue": 0, + "avoided_harm": 16, + "selected_both_correct": 651, + "selected_both_wrong": 11 + }, + { + "budget": 0.99, + "threshold": -0.9774226546287537, + "operator": "ge", + "actual_fraction": 0.99, + "metric": 0.5575, + "rescued": 2, + "harmed": 520, + "missed_rescue": 0, + "avoided_harm": 8, + "selected_both_correct": 655, + "selected_both_wrong": 11 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5508333333333333, + "rescued": 2, + "harmed": 528, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 659, + "selected_both_wrong": 11 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9016223679668622, + "rescued": 0, + "harmed": 0, + "missed_rescue": 144, + "avoided_harm": 2192, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.08698653779772178, + "metric": 0.8878149810148429, + "rescued": 70, + "harmed": 150, + "missed_rescue": 74, + "avoided_harm": 2042, + "selected_both_correct": 85, + "selected_both_wrong": 199 + }, + { + "budget": 0.02, + "threshold": -0.6014261841773987, + "operator": "ge", + "actual_fraction": 0.13910942354159475, + "metric": 0.8683120469451157, + "rescued": 93, + "harmed": 286, + "missed_rescue": 51, + "avoided_harm": 1906, + "selected_both_correct": 168, + "selected_both_wrong": 259 + }, + { + "budget": 0.03, + "threshold": -0.6401472687721252, + "operator": "ge", + "actual_fraction": 0.15947531929582326, + "metric": 0.8596824301001036, + "rescued": 100, + "harmed": 343, + "missed_rescue": 44, + "avoided_harm": 1849, + "selected_both_correct": 203, + "selected_both_wrong": 278 + }, + { + "budget": 0.04, + "threshold": -0.6884650588035583, + "operator": "ge", + "actual_fraction": 0.1886434242319641, + "metric": 0.845702450811184, + "rescued": 106, + "harmed": 430, + "missed_rescue": 38, + "avoided_harm": 1762, + "selected_both_correct": 258, + "selected_both_wrong": 299 + }, + { + "budget": 0.05, + "threshold": -0.7208121418952942, + "operator": "ge", + "actual_fraction": 0.2143596824301001, + "metric": 0.8329306178805661, + "rescued": 111, + "harmed": 509, + "missed_rescue": 33, + "avoided_harm": 1683, + "selected_both_correct": 305, + "selected_both_wrong": 317 + }, + { + "budget": 0.06, + "threshold": -0.7578181624412537, + "operator": "ge", + "actual_fraction": 0.24525371073524335, + "metric": 0.819986192613048, + "rescued": 119, + "harmed": 592, + "missed_rescue": 25, + "avoided_harm": 1600, + "selected_both_correct": 384, + "selected_both_wrong": 326 + }, + { + "budget": 0.07, + "threshold": -0.7790470719337463, + "operator": "ge", + "actual_fraction": 0.26734552985847426, + "metric": 0.8087676907145323, + "rescued": 122, + "harmed": 660, + "missed_rescue": 22, + "avoided_harm": 1532, + "selected_both_correct": 430, + "selected_both_wrong": 337 + }, + { + "budget": 0.08, + "threshold": -0.7927228808403015, + "operator": "ge", + "actual_fraction": 0.2827062478425958, + "metric": 0.8015188125647221, + "rescued": 125, + "harmed": 705, + "missed_rescue": 19, + "avoided_harm": 1487, + "selected_both_correct": 468, + "selected_both_wrong": 340 + }, + { + "budget": 0.09, + "threshold": -0.8049252033233643, + "operator": "ge", + "actual_fraction": 0.2999654815326199, + "metric": 0.7951328960994132, + "rescued": 127, + "harmed": 744, + "missed_rescue": 17, + "avoided_harm": 1448, + "selected_both_correct": 521, + "selected_both_wrong": 346 + }, + { + "budget": 0.1, + "threshold": -0.80850750207901, + "operator": "ge", + "actual_fraction": 0.30531584397652745, + "metric": 0.793579565067311, + "rescued": 127, + "harmed": 753, + "missed_rescue": 17, + "avoided_harm": 1439, + "selected_both_correct": 541, + "selected_both_wrong": 348 + }, + { + "budget": 0.11, + "threshold": -0.819871723651886, + "operator": "ge", + "actual_fraction": 0.3258543320676562, + "metric": 0.7839143942008975, + "rescued": 129, + "harmed": 811, + "missed_rescue": 15, + "avoided_harm": 1381, + "selected_both_correct": 593, + "selected_both_wrong": 355 + }, + { + "budget": 0.12, + "threshold": -0.8310170769691467, + "operator": "ge", + "actual_fraction": 0.35053503624439075, + "metric": 0.7733862616499827, + "rescued": 132, + "harmed": 875, + "missed_rescue": 12, + "avoided_harm": 1317, + "selected_both_correct": 662, + "selected_both_wrong": 362 + }, + { + "budget": 0.13, + "threshold": -0.8376906514167786, + "operator": "ge", + "actual_fraction": 0.36796686227131514, + "metric": 0.7654470141525717, + "rescued": 133, + "harmed": 922, + "missed_rescue": 11, + "avoided_harm": 1270, + "selected_both_correct": 712, + "selected_both_wrong": 365 + }, + { + "budget": 0.14, + "threshold": -0.8419997692108154, + "operator": "ge", + "actual_fraction": 0.3798757335174318, + "metric": 0.7609596133931653, + "rescued": 133, + "harmed": 948, + "missed_rescue": 11, + "avoided_harm": 1244, + "selected_both_correct": 753, + "selected_both_wrong": 367 + }, + { + "budget": 0.15, + "threshold": -0.8447275161743164, + "operator": "ge", + "actual_fraction": 0.3867794269934415, + "metric": 0.7581981360027614, + "rescued": 133, + "harmed": 964, + "missed_rescue": 11, + "avoided_harm": 1228, + "selected_both_correct": 777, + "selected_both_wrong": 367 + }, + { + "budget": 0.16, + "threshold": -0.8506776690483093, + "operator": "ge", + "actual_fraction": 0.4050742147048671, + "metric": 0.7509492578529513, + "rescued": 135, + "harmed": 1008, + "missed_rescue": 9, + "avoided_harm": 1184, + "selected_both_correct": 835, + "selected_both_wrong": 369 + }, + { + "budget": 0.17, + "threshold": -0.8545639514923096, + "operator": "ge", + "actual_fraction": 0.4175008629616845, + "metric": 0.7462892647566448, + "rescued": 135, + "harmed": 1035, + "missed_rescue": 9, + "avoided_harm": 1157, + "selected_both_correct": 876, + "selected_both_wrong": 373 + }, + { + "budget": 0.18, + "threshold": -0.8578683137893677, + "operator": "ge", + "actual_fraction": 0.4290645495340007, + "metric": 0.74283741801864, + "rescued": 136, + "harmed": 1056, + "missed_rescue": 8, + "avoided_harm": 1136, + "selected_both_correct": 920, + "selected_both_wrong": 374 + }, + { + "budget": 0.19, + "threshold": -0.86090087890625, + "operator": "ge", + "actual_fraction": 0.43993786675871593, + "metric": 0.7399033482913359, + "rescued": 137, + "harmed": 1074, + "missed_rescue": 7, + "avoided_harm": 1118, + "selected_both_correct": 962, + "selected_both_wrong": 376 + }, + { + "budget": 0.2, + "threshold": -0.8640955090522766, + "operator": "ge", + "actual_fraction": 0.4520193303417328, + "metric": 0.7347255781843286, + "rescued": 138, + "harmed": 1105, + "missed_rescue": 6, + "avoided_harm": 1087, + "selected_both_correct": 1000, + "selected_both_wrong": 376 + }, + { + "budget": 0.21, + "threshold": -0.8664237260818481, + "operator": "ge", + "actual_fraction": 0.46047635484984467, + "metric": 0.731446323783224, + "rescued": 138, + "harmed": 1124, + "missed_rescue": 6, + "avoided_harm": 1068, + "selected_both_correct": 1028, + "selected_both_wrong": 378 + }, + { + "budget": 0.22, + "threshold": -0.8686786890029907, + "operator": "ge", + "actual_fraction": 0.46945115636865725, + "metric": 0.7279944770452192, + "rescued": 138, + "harmed": 1144, + "missed_rescue": 6, + "avoided_harm": 1048, + "selected_both_correct": 1060, + "selected_both_wrong": 378 + }, + { + "budget": 0.23, + "threshold": -0.8709638118743896, + "operator": "ge", + "actual_fraction": 0.48066965826717295, + "metric": 0.7241974456334139, + "rescued": 138, + "harmed": 1166, + "missed_rescue": 6, + "avoided_harm": 1026, + "selected_both_correct": 1102, + "selected_both_wrong": 379 + }, + { + "budget": 0.24, + "threshold": -0.8724074959754944, + "operator": "ge", + "actual_fraction": 0.48912668277528476, + "metric": 0.7210907835692095, + "rescued": 138, + "harmed": 1184, + "missed_rescue": 6, + "avoided_harm": 1008, + "selected_both_correct": 1132, + "selected_both_wrong": 380 + }, + { + "budget": 0.25, + "threshold": -0.8736594319343567, + "operator": "ge", + "actual_fraction": 0.49413186054539177, + "metric": 0.7200552295478081, + "rescued": 138, + "harmed": 1190, + "missed_rescue": 6, + "avoided_harm": 1002, + "selected_both_correct": 1154, + "selected_both_wrong": 381 + }, + { + "budget": 0.26, + "threshold": -0.8752554059028625, + "operator": "ge", + "actual_fraction": 0.5005177770107008, + "metric": 0.717811529168105, + "rescued": 138, + "harmed": 1203, + "missed_rescue": 6, + "avoided_harm": 989, + "selected_both_correct": 1178, + "selected_both_wrong": 381 + }, + { + "budget": 0.27, + "threshold": -0.8766024112701416, + "operator": "ge", + "actual_fraction": 0.5081118398343114, + "metric": 0.7153952364515016, + "rescued": 138, + "harmed": 1217, + "missed_rescue": 6, + "avoided_harm": 975, + "selected_both_correct": 1207, + "selected_both_wrong": 382 + }, + { + "budget": 0.28, + "threshold": -0.878311276435852, + "operator": "ge", + "actual_fraction": 0.5169140490162237, + "metric": 0.7131515360717984, + "rescued": 138, + "harmed": 1230, + "missed_rescue": 6, + "avoided_harm": 962, + "selected_both_correct": 1243, + "selected_both_wrong": 384 + }, + { + "budget": 0.29, + "threshold": -0.880103588104248, + "operator": "ge", + "actual_fraction": 0.5277873662409389, + "metric": 0.7105626510182947, + "rescued": 138, + "harmed": 1245, + "missed_rescue": 6, + "avoided_harm": 947, + "selected_both_correct": 1290, + "selected_both_wrong": 385 + }, + { + "budget": 0.3, + "threshold": -0.8824952244758606, + "operator": "ge", + "actual_fraction": 0.5419399378667588, + "metric": 0.704349326889886, + "rescued": 138, + "harmed": 1281, + "missed_rescue": 6, + "avoided_harm": 911, + "selected_both_correct": 1336, + "selected_both_wrong": 385 + }, + { + "budget": 0.31, + "threshold": -0.8831692337989807, + "operator": "ge", + "actual_fraction": 0.5453917846047636, + "metric": 0.7034863652053849, + "rescued": 138, + "harmed": 1286, + "missed_rescue": 6, + "avoided_harm": 906, + "selected_both_correct": 1350, + "selected_both_wrong": 386 + }, + { + "budget": 0.32, + "threshold": -0.8845130205154419, + "operator": "ge", + "actual_fraction": 0.5566102865032793, + "metric": 0.7003797031411805, + "rescued": 138, + "harmed": 1304, + "missed_rescue": 6, + "avoided_harm": 888, + "selected_both_correct": 1394, + "selected_both_wrong": 389 + }, + { + "budget": 0.33, + "threshold": -0.8855781555175781, + "operator": "ge", + "actual_fraction": 0.5633413876423887, + "metric": 0.6976182257507767, + "rescued": 138, + "harmed": 1320, + "missed_rescue": 6, + "avoided_harm": 872, + "selected_both_correct": 1417, + "selected_both_wrong": 389 + }, + { + "budget": 0.34, + "threshold": -0.8865919709205627, + "operator": "ge", + "actual_fraction": 0.5716258198136003, + "metric": 0.6958923023817742, + "rescued": 138, + "harmed": 1330, + "missed_rescue": 6, + "avoided_harm": 862, + "selected_both_correct": 1454, + "selected_both_wrong": 390 + }, + { + "budget": 0.35, + "threshold": -0.8877584338188171, + "operator": "ge", + "actual_fraction": 0.5819813600276148, + "metric": 0.6929582326544701, + "rescued": 139, + "harmed": 1348, + "missed_rescue": 5, + "avoided_harm": 844, + "selected_both_correct": 1494, + "selected_both_wrong": 391 + }, + { + "budget": 0.36, + "threshold": -0.8890330195426941, + "operator": "ge", + "actual_fraction": 0.5902657921988264, + "metric": 0.6910597169485675, + "rescued": 139, + "harmed": 1359, + "missed_rescue": 5, + "avoided_harm": 833, + "selected_both_correct": 1529, + "selected_both_wrong": 393 + }, + { + "budget": 0.37, + "threshold": -0.890850305557251, + "operator": "ge", + "actual_fraction": 0.6047635484984467, + "metric": 0.6853641698308595, + "rescued": 139, + "harmed": 1392, + "missed_rescue": 5, + "avoided_harm": 800, + "selected_both_correct": 1580, + "selected_both_wrong": 393 + }, + { + "budget": 0.38, + "threshold": -0.8916605710983276, + "operator": "ge", + "actual_fraction": 0.6113220573006558, + "metric": 0.6834656541249569, + "rescued": 139, + "harmed": 1403, + "missed_rescue": 5, + "avoided_harm": 789, + "selected_both_correct": 1607, + "selected_both_wrong": 393 + }, + { + "budget": 0.39, + "threshold": -0.8928589224815369, + "operator": "ge", + "actual_fraction": 0.6206420434932689, + "metric": 0.681394546082154, + "rescued": 140, + "harmed": 1416, + "missed_rescue": 4, + "avoided_harm": 776, + "selected_both_correct": 1647, + "selected_both_wrong": 393 + }, + { + "budget": 0.4, + "threshold": -0.8940943479537964, + "operator": "ge", + "actual_fraction": 0.6289264756644805, + "metric": 0.6789782533655506, + "rescued": 140, + "harmed": 1430, + "missed_rescue": 4, + "avoided_harm": 762, + "selected_both_correct": 1681, + "selected_both_wrong": 393 + }, + { + "budget": 0.41, + "threshold": -0.8949463963508606, + "operator": "ge", + "actual_fraction": 0.6370383154987919, + "metric": 0.6762167759751467, + "rescued": 140, + "harmed": 1446, + "missed_rescue": 4, + "avoided_harm": 746, + "selected_both_correct": 1711, + "selected_both_wrong": 394 + }, + { + "budget": 0.42, + "threshold": -0.8960229158401489, + "operator": "ge", + "actual_fraction": 0.6435968243010011, + "metric": 0.6739730755954436, + "rescued": 140, + "harmed": 1459, + "missed_rescue": 4, + "avoided_harm": 733, + "selected_both_correct": 1734, + "selected_both_wrong": 396 + }, + { + "budget": 0.43, + "threshold": -0.8966536521911621, + "operator": "ge", + "actual_fraction": 0.6491197790818087, + "metric": 0.6724197445633414, + "rescued": 140, + "harmed": 1468, + "missed_rescue": 4, + "avoided_harm": 724, + "selected_both_correct": 1757, + "selected_both_wrong": 396 + }, + { + "budget": 0.44, + "threshold": -0.8975363373756409, + "operator": "ge", + "actual_fraction": 0.6558508802209182, + "metric": 0.670003451846738, + "rescued": 140, + "harmed": 1482, + "missed_rescue": 4, + "avoided_harm": 710, + "selected_both_correct": 1782, + "selected_both_wrong": 396 + }, + { + "budget": 0.45, + "threshold": -0.8988122940063477, + "operator": "ge", + "actual_fraction": 0.6641353123921298, + "metric": 0.6681049361408353, + "rescued": 140, + "harmed": 1493, + "missed_rescue": 4, + "avoided_harm": 699, + "selected_both_correct": 1819, + "selected_both_wrong": 396 + }, + { + "budget": 0.46, + "threshold": -0.9000959396362305, + "operator": "ge", + "actual_fraction": 0.6739730755954436, + "metric": 0.6644804970659303, + "rescued": 140, + "harmed": 1514, + "missed_rescue": 4, + "avoided_harm": 678, + "selected_both_correct": 1855, + "selected_both_wrong": 396 + }, + { + "budget": 0.47, + "threshold": -0.9013051986694336, + "operator": "ge", + "actual_fraction": 0.6848463928201588, + "metric": 0.6596479116327235, + "rescued": 141, + "harmed": 1543, + "missed_rescue": 3, + "avoided_harm": 649, + "selected_both_correct": 1887, + "selected_both_wrong": 397 + }, + { + "budget": 0.48, + "threshold": -0.9029324054718018, + "operator": "ge", + "actual_fraction": 0.696410079392475, + "metric": 0.6575768035899207, + "rescued": 141, + "harmed": 1555, + "missed_rescue": 3, + "avoided_harm": 637, + "selected_both_correct": 1941, + "selected_both_wrong": 398 + }, + { + "budget": 0.49, + "threshold": -0.9036871790885925, + "operator": "ge", + "actual_fraction": 0.7026234035208837, + "metric": 0.6561960648947187, + "rescued": 141, + "harmed": 1563, + "missed_rescue": 3, + "avoided_harm": 629, + "selected_both_correct": 1967, + "selected_both_wrong": 400 + }, + { + "budget": 0.5, + "threshold": -0.9044621586799622, + "operator": "ge", + "actual_fraction": 0.7088367276492924, + "metric": 0.654297549188816, + "rescued": 141, + "harmed": 1574, + "missed_rescue": 3, + "avoided_harm": 618, + "selected_both_correct": 1992, + "selected_both_wrong": 400 + }, + { + "budget": 0.51, + "threshold": -0.9054575562477112, + "operator": "ge", + "actual_fraction": 0.717121159820504, + "metric": 0.6525716258198136, + "rescued": 141, + "harmed": 1584, + "missed_rescue": 3, + "avoided_harm": 608, + "selected_both_correct": 2030, + "selected_both_wrong": 400 + }, + { + "budget": 0.52, + "threshold": -0.906682550907135, + "operator": "ge", + "actual_fraction": 0.7262685536762168, + "metric": 0.64998274076631, + "rescued": 141, + "harmed": 1599, + "missed_rescue": 3, + "avoided_harm": 593, + "selected_both_correct": 2068, + "selected_both_wrong": 400 + }, + { + "budget": 0.53, + "threshold": -0.9072114825248718, + "operator": "ge", + "actual_fraction": 0.7300655850880221, + "metric": 0.6484294097342078, + "rescued": 141, + "harmed": 1608, + "missed_rescue": 3, + "avoided_harm": 584, + "selected_both_correct": 2080, + "selected_both_wrong": 401 + }, + { + "budget": 0.54, + "threshold": -0.9083563089370728, + "operator": "ge", + "actual_fraction": 0.7381774249223334, + "metric": 0.6460131170176044, + "rescued": 141, + "harmed": 1622, + "missed_rescue": 3, + "avoided_harm": 570, + "selected_both_correct": 2112, + "selected_both_wrong": 402 + }, + { + "budget": 0.55, + "threshold": -0.9095192551612854, + "operator": "ge", + "actual_fraction": 0.7487055574732482, + "metric": 0.6435968243010011, + "rescued": 141, + "harmed": 1636, + "missed_rescue": 3, + "avoided_harm": 556, + "selected_both_correct": 2157, + "selected_both_wrong": 404 + }, + { + "budget": 0.56, + "threshold": -0.9106979370117188, + "operator": "ge", + "actual_fraction": 0.759233690024163, + "metric": 0.6394546082153952, + "rescued": 141, + "harmed": 1660, + "missed_rescue": 3, + "avoided_harm": 532, + "selected_both_correct": 2193, + "selected_both_wrong": 405 + }, + { + "budget": 0.57, + "threshold": -0.9115362167358398, + "operator": "ge", + "actual_fraction": 0.767863306869175, + "metric": 0.6372109078356921, + "rescued": 141, + "harmed": 1673, + "missed_rescue": 3, + "avoided_harm": 519, + "selected_both_correct": 2230, + "selected_both_wrong": 405 + }, + { + "budget": 0.58, + "threshold": -0.9124338030815125, + "operator": "ge", + "actual_fraction": 0.7751121850189852, + "metric": 0.6339316534345875, + "rescued": 141, + "harmed": 1692, + "missed_rescue": 3, + "avoided_harm": 500, + "selected_both_correct": 2253, + "selected_both_wrong": 405 + }, + { + "budget": 0.59, + "threshold": -0.9134801626205444, + "operator": "ge", + "actual_fraction": 0.7830514325163963, + "metric": 0.6316879530548843, + "rescued": 141, + "harmed": 1705, + "missed_rescue": 3, + "avoided_harm": 487, + "selected_both_correct": 2286, + "selected_both_wrong": 405 + }, + { + "budget": 0.6, + "threshold": -0.9143747091293335, + "operator": "ge", + "actual_fraction": 0.7904729030031067, + "metric": 0.629271660338281, + "rescued": 141, + "harmed": 1719, + "missed_rescue": 3, + "avoided_harm": 473, + "selected_both_correct": 2314, + "selected_both_wrong": 406 + }, + { + "budget": 0.61, + "threshold": -0.915804386138916, + "operator": "ge", + "actual_fraction": 0.8010010355540214, + "metric": 0.6272005522954781, + "rescued": 141, + "harmed": 1731, + "missed_rescue": 3, + "avoided_harm": 461, + "selected_both_correct": 2361, + "selected_both_wrong": 408 + }, + { + "budget": 0.62, + "threshold": -0.9165146350860596, + "operator": "ge", + "actual_fraction": 0.8063513979979289, + "metric": 0.6253020365895754, + "rescued": 141, + "harmed": 1742, + "missed_rescue": 3, + "avoided_harm": 450, + "selected_both_correct": 2381, + "selected_both_wrong": 408 + }, + { + "budget": 0.63, + "threshold": -0.9172822833061218, + "operator": "ge", + "actual_fraction": 0.8110113910942354, + "metric": 0.6240938902312737, + "rescued": 141, + "harmed": 1749, + "missed_rescue": 3, + "avoided_harm": 443, + "selected_both_correct": 2401, + "selected_both_wrong": 408 + }, + { + "budget": 0.64, + "threshold": -0.9181071519851685, + "operator": "ge", + "actual_fraction": 0.8172247152226441, + "metric": 0.6220227821884708, + "rescued": 141, + "harmed": 1761, + "missed_rescue": 3, + "avoided_harm": 431, + "selected_both_correct": 2424, + "selected_both_wrong": 409 + }, + { + "budget": 0.65, + "threshold": -0.9192029237747192, + "operator": "ge", + "actual_fraction": 0.824301001035554, + "metric": 0.619951674145668, + "rescued": 141, + "harmed": 1773, + "missed_rescue": 3, + "avoided_harm": 419, + "selected_both_correct": 2452, + "selected_both_wrong": 410 + }, + { + "budget": 0.66, + "threshold": -0.9200543761253357, + "operator": "ge", + "actual_fraction": 0.829306178805661, + "metric": 0.618570935450466, + "rescued": 141, + "harmed": 1781, + "missed_rescue": 3, + "avoided_harm": 411, + "selected_both_correct": 2473, + "selected_both_wrong": 410 + }, + { + "budget": 0.67, + "threshold": -0.9208071231842041, + "operator": "ge", + "actual_fraction": 0.8348291335864687, + "metric": 0.6164998274076631, + "rescued": 141, + "harmed": 1793, + "missed_rescue": 3, + "avoided_harm": 399, + "selected_both_correct": 2493, + "selected_both_wrong": 410 + }, + { + "budget": 0.68, + "threshold": -0.9215553998947144, + "operator": "ge", + "actual_fraction": 0.840006903693476, + "metric": 0.6151190887124611, + "rescued": 141, + "harmed": 1801, + "missed_rescue": 3, + "avoided_harm": 391, + "selected_both_correct": 2515, + "selected_both_wrong": 410 + }, + { + "budget": 0.69, + "threshold": -0.9220835566520691, + "operator": "ge", + "actual_fraction": 0.8443217121159821, + "metric": 0.613565757680359, + "rescued": 141, + "harmed": 1810, + "missed_rescue": 3, + "avoided_harm": 382, + "selected_both_correct": 2530, + "selected_both_wrong": 411 + }, + { + "budget": 0.7, + "threshold": -0.9232237935066223, + "operator": "ge", + "actual_fraction": 0.852088367276493, + "metric": 0.6113220573006558, + "rescued": 141, + "harmed": 1823, + "missed_rescue": 3, + "avoided_harm": 369, + "selected_both_correct": 2560, + "selected_both_wrong": 413 + }, + { + "budget": 0.71, + "threshold": -0.9243364334106445, + "operator": "ge", + "actual_fraction": 0.8598550224370038, + "metric": 0.6082153952364515, + "rescued": 141, + "harmed": 1841, + "missed_rescue": 3, + "avoided_harm": 351, + "selected_both_correct": 2585, + "selected_both_wrong": 415 + }, + { + "budget": 0.72, + "threshold": -0.9252541065216064, + "operator": "ge", + "actual_fraction": 0.8641698308595098, + "metric": 0.6071798412150501, + "rescued": 141, + "harmed": 1847, + "missed_rescue": 3, + "avoided_harm": 345, + "selected_both_correct": 2604, + "selected_both_wrong": 415 + }, + { + "budget": 0.73, + "threshold": -0.925781786441803, + "operator": "ge", + "actual_fraction": 0.8681394546082154, + "metric": 0.6057991025198481, + "rescued": 141, + "harmed": 1855, + "missed_rescue": 3, + "avoided_harm": 337, + "selected_both_correct": 2619, + "selected_both_wrong": 415 + }, + { + "budget": 0.74, + "threshold": -0.9274963140487671, + "operator": "ge", + "actual_fraction": 0.877114256127028, + "metric": 0.6018294787711426, + "rescued": 141, + "harmed": 1878, + "missed_rescue": 3, + "avoided_harm": 314, + "selected_both_correct": 2648, + "selected_both_wrong": 415 + }, + { + "budget": 0.75, + "threshold": -0.9286423325538635, + "operator": "ge", + "actual_fraction": 0.8828098032447359, + "metric": 0.5995857783914395, + "rescued": 141, + "harmed": 1891, + "missed_rescue": 3, + "avoided_harm": 301, + "selected_both_correct": 2667, + "selected_both_wrong": 416 + }, + { + "budget": 0.76, + "threshold": -0.9301291704177856, + "operator": "ge", + "actual_fraction": 0.8900586813945461, + "metric": 0.5969968933379358, + "rescued": 141, + "harmed": 1906, + "missed_rescue": 3, + "avoided_harm": 286, + "selected_both_correct": 2693, + "selected_both_wrong": 417 + }, + { + "budget": 0.77, + "threshold": -0.9318777918815613, + "operator": "ge", + "actual_fraction": 0.8979979288919572, + "metric": 0.5945806006213324, + "rescued": 141, + "harmed": 1920, + "missed_rescue": 3, + "avoided_harm": 272, + "selected_both_correct": 2725, + "selected_both_wrong": 417 + }, + { + "budget": 0.78, + "threshold": -0.9332055449485779, + "operator": "ge", + "actual_fraction": 0.9052468070417673, + "metric": 0.5916465308940283, + "rescued": 141, + "harmed": 1937, + "missed_rescue": 3, + "avoided_harm": 255, + "selected_both_correct": 2750, + "selected_both_wrong": 417 + }, + { + "budget": 0.79, + "threshold": -0.9344666600227356, + "operator": "ge", + "actual_fraction": 0.9100793924749742, + "metric": 0.5888850535036244, + "rescued": 141, + "harmed": 1953, + "missed_rescue": 3, + "avoided_harm": 239, + "selected_both_correct": 2762, + "selected_both_wrong": 417 + }, + { + "budget": 0.8, + "threshold": -0.9362465143203735, + "operator": "ge", + "actual_fraction": 0.917155678287884, + "metric": 0.5862961684501208, + "rescued": 141, + "harmed": 1968, + "missed_rescue": 3, + "avoided_harm": 224, + "selected_both_correct": 2787, + "selected_both_wrong": 418 + }, + { + "budget": 0.81, + "threshold": -0.937842071056366, + "operator": "ge", + "actual_fraction": 0.9237141870900932, + "metric": 0.5830169140490162, + "rescued": 141, + "harmed": 1987, + "missed_rescue": 3, + "avoided_harm": 205, + "selected_both_correct": 2806, + "selected_both_wrong": 418 + }, + { + "budget": 0.82, + "threshold": -0.9384457468986511, + "operator": "ge", + "actual_fraction": 0.9249223334483949, + "metric": 0.5823265447014152, + "rescued": 141, + "harmed": 1991, + "missed_rescue": 3, + "avoided_harm": 201, + "selected_both_correct": 2808, + "selected_both_wrong": 419 + }, + { + "budget": 0.83, + "threshold": -0.9396296143531799, + "operator": "ge", + "actual_fraction": 0.927166033828098, + "metric": 0.5819813600276148, + "rescued": 142, + "harmed": 1994, + "missed_rescue": 2, + "avoided_harm": 198, + "selected_both_correct": 2817, + "selected_both_wrong": 419 + }, + { + "budget": 0.84, + "threshold": -0.9407488703727722, + "operator": "ge", + "actual_fraction": 0.9316534345875043, + "metric": 0.5802554366586123, + "rescued": 142, + "harmed": 2004, + "missed_rescue": 2, + "avoided_harm": 188, + "selected_both_correct": 2833, + "selected_both_wrong": 419 + }, + { + "budget": 0.85, + "threshold": -0.942738950252533, + "operator": "ge", + "actual_fraction": 0.9397652744218157, + "metric": 0.576458405246807, + "rescued": 142, + "harmed": 2026, + "missed_rescue": 2, + "avoided_harm": 166, + "selected_both_correct": 2858, + "selected_both_wrong": 419 + }, + { + "budget": 0.86, + "threshold": -0.9438527226448059, + "operator": "gt", + "actual_fraction": 0.9432171211598205, + "metric": 0.5749050742147048, + "rescued": 142, + "harmed": 2035, + "missed_rescue": 2, + "avoided_harm": 157, + "selected_both_correct": 2868, + "selected_both_wrong": 420 + }, + { + "budget": 0.87, + "threshold": -0.9450756907463074, + "operator": "ge", + "actual_fraction": 0.9456334138764239, + "metric": 0.5742147048671039, + "rescued": 142, + "harmed": 2039, + "missed_rescue": 2, + "avoided_harm": 153, + "selected_both_correct": 2878, + "selected_both_wrong": 420 + }, + { + "budget": 0.88, + "threshold": -0.9481598138809204, + "operator": "ge", + "actual_fraction": 0.9539178460476355, + "metric": 0.5704176734552986, + "rescued": 142, + "harmed": 2061, + "missed_rescue": 2, + "avoided_harm": 131, + "selected_both_correct": 2902, + "selected_both_wrong": 422 + }, + { + "budget": 0.89, + "threshold": -0.9500836730003357, + "operator": "ge", + "actual_fraction": 0.9606489471867449, + "metric": 0.5676561960648947, + "rescued": 142, + "harmed": 2077, + "missed_rescue": 2, + "avoided_harm": 115, + "selected_both_correct": 2924, + "selected_both_wrong": 423 + }, + { + "budget": 0.9, + "threshold": -0.9519118666648865, + "operator": "ge", + "actual_fraction": 0.9647911632723507, + "metric": 0.5662754573696928, + "rescued": 143, + "harmed": 2086, + "missed_rescue": 1, + "avoided_harm": 106, + "selected_both_correct": 2938, + "selected_both_wrong": 423 + }, + { + "budget": 0.91, + "threshold": -0.9543898701667786, + "operator": "ge", + "actual_fraction": 0.9697963410424577, + "metric": 0.5636865723161891, + "rescued": 143, + "harmed": 2101, + "missed_rescue": 1, + "avoided_harm": 91, + "selected_both_correct": 2951, + "selected_both_wrong": 424 + }, + { + "budget": 0.92, + "threshold": -0.9570822715759277, + "operator": "ge", + "actual_fraction": 0.9760096651708664, + "metric": 0.5605799102519848, + "rescued": 143, + "harmed": 2119, + "missed_rescue": 1, + "avoided_harm": 73, + "selected_both_correct": 2968, + "selected_both_wrong": 425 + }, + { + "budget": 0.93, + "threshold": -0.9593446254730225, + "operator": "ge", + "actual_fraction": 0.9801518812564722, + "metric": 0.5586813945460821, + "rescued": 143, + "harmed": 2130, + "missed_rescue": 1, + "avoided_harm": 62, + "selected_both_correct": 2981, + "selected_both_wrong": 425 + }, + { + "budget": 0.94, + "threshold": -0.9610995054244995, + "operator": "ge", + "actual_fraction": 0.9817052122885744, + "metric": 0.5579910251984812, + "rescued": 143, + "harmed": 2134, + "missed_rescue": 1, + "avoided_harm": 58, + "selected_both_correct": 2986, + "selected_both_wrong": 425 + }, + { + "budget": 0.95, + "threshold": -0.9635111093521118, + "operator": "ge", + "actual_fraction": 0.9853296513634795, + "metric": 0.556437694166379, + "rescued": 143, + "harmed": 2143, + "missed_rescue": 1, + "avoided_harm": 49, + "selected_both_correct": 2998, + "selected_both_wrong": 425 + }, + { + "budget": 0.96, + "threshold": -0.9648061990737915, + "operator": "ge", + "actual_fraction": 0.9875733517431826, + "metric": 0.5548843631342768, + "rescued": 143, + "harmed": 2152, + "missed_rescue": 1, + "avoided_harm": 40, + "selected_both_correct": 3002, + "selected_both_wrong": 425 + }, + { + "budget": 0.97, + "threshold": -0.9674390554428101, + "operator": "ge", + "actual_fraction": 0.9906800138073869, + "metric": 0.5533310321021747, + "rescued": 143, + "harmed": 2161, + "missed_rescue": 1, + "avoided_harm": 31, + "selected_both_correct": 3011, + "selected_both_wrong": 425 + }, + { + "budget": 0.98, + "threshold": -0.9724470973014832, + "operator": "ge", + "actual_fraction": 0.9937866758715913, + "metric": 0.5516051087331723, + "rescued": 143, + "harmed": 2171, + "missed_rescue": 1, + "avoided_harm": 21, + "selected_both_correct": 3018, + "selected_both_wrong": 426 + }, + { + "budget": 0.99, + "threshold": -0.9774226546287537, + "operator": "ge", + "actual_fraction": 0.9965481532619952, + "metric": 0.5502243700379703, + "rescued": 144, + "harmed": 2180, + "missed_rescue": 0, + "avoided_harm": 12, + "selected_both_correct": 3024, + "selected_both_wrong": 426 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5481532619951675, + "rescued": 144, + "harmed": 2192, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 3032, + "selected_both_wrong": 426 + } + ] + }, + "learned": { + "selected_budget_index": 1, + "validation_area_0_25": 0.97575, + "validation_point": { + "budget": 0.01, + "threshold": 4.029286861419678, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9883333333333333, + "rescued": 0, + "harmed": 1, + "missed_rescue": 2, + "avoided_harm": 527, + "selected_both_correct": 11, + "selected_both_wrong": 0 + }, + "test_point": { + "budget": 0.01, + "threshold": 4.029286861419678, + "operator": "ge", + "actual_fraction": 0.016568864342423197, + "metric": 0.9005868139454608, + "rescued": 0, + "harmed": 6, + "missed_rescue": 144, + "avoided_harm": 2186, + "selected_both_correct": 86, + "selected_both_wrong": 4 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9891666666666666, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2, + "avoided_harm": 528, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 4.029286861419678, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9883333333333333, + "rescued": 0, + "harmed": 1, + "missed_rescue": 2, + "avoided_harm": 527, + "selected_both_correct": 11, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": 2.9611759185791016, + "operator": "ge", + "actual_fraction": 0.02, + "metric": 0.9883333333333333, + "rescued": 0, + "harmed": 1, + "missed_rescue": 2, + "avoided_harm": 527, + "selected_both_correct": 23, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": 1.9980177879333496, + "operator": "ge", + "actual_fraction": 0.03, + "metric": 0.9875, + "rescued": 0, + "harmed": 2, + "missed_rescue": 2, + "avoided_harm": 526, + "selected_both_correct": 34, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": 1.3639881610870361, + "operator": "ge", + "actual_fraction": 0.04, + "metric": 0.9866666666666667, + "rescued": 0, + "harmed": 3, + "missed_rescue": 2, + "avoided_harm": 525, + "selected_both_correct": 45, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": 0.9321551322937012, + "operator": "ge", + "actual_fraction": 0.05, + "metric": 0.985, + "rescued": 0, + "harmed": 5, + "missed_rescue": 2, + "avoided_harm": 523, + "selected_both_correct": 55, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": 0.3624706268310547, + "operator": "ge", + "actual_fraction": 0.06, + "metric": 0.985, + "rescued": 0, + "harmed": 5, + "missed_rescue": 2, + "avoided_harm": 523, + "selected_both_correct": 67, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": -0.07812309265136719, + "operator": "ge", + "actual_fraction": 0.07, + "metric": 0.985, + "rescued": 0, + "harmed": 5, + "missed_rescue": 2, + "avoided_harm": 523, + "selected_both_correct": 79, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": -0.7457926273345947, + "operator": "ge", + "actual_fraction": 0.08, + "metric": 0.9841666666666666, + "rescued": 0, + "harmed": 6, + "missed_rescue": 2, + "avoided_harm": 522, + "selected_both_correct": 89, + "selected_both_wrong": 1 + }, + { + "budget": 0.09, + "threshold": -1.2083253860473633, + "operator": "ge", + "actual_fraction": 0.09, + "metric": 0.9841666666666666, + "rescued": 0, + "harmed": 6, + "missed_rescue": 2, + "avoided_harm": 522, + "selected_both_correct": 100, + "selected_both_wrong": 2 + }, + { + "budget": 0.1, + "threshold": -1.6223034858703613, + "operator": "ge", + "actual_fraction": 0.1, + "metric": 0.9825, + "rescued": 0, + "harmed": 8, + "missed_rescue": 2, + "avoided_harm": 520, + "selected_both_correct": 110, + "selected_both_wrong": 2 + }, + { + "budget": 0.11, + "threshold": -2.024738073348999, + "operator": "ge", + "actual_fraction": 0.11, + "metric": 0.98, + "rescued": 0, + "harmed": 11, + "missed_rescue": 2, + "avoided_harm": 517, + "selected_both_correct": 119, + "selected_both_wrong": 2 + }, + { + "budget": 0.12, + "threshold": -2.3822429180145264, + "operator": "ge", + "actual_fraction": 0.12, + "metric": 0.98, + "rescued": 0, + "harmed": 11, + "missed_rescue": 2, + "avoided_harm": 517, + "selected_both_correct": 131, + "selected_both_wrong": 2 + }, + { + "budget": 0.13, + "threshold": -2.7217001914978027, + "operator": "ge", + "actual_fraction": 0.13, + "metric": 0.9791666666666666, + "rescued": 0, + "harmed": 12, + "missed_rescue": 2, + "avoided_harm": 516, + "selected_both_correct": 142, + "selected_both_wrong": 2 + }, + { + "budget": 0.14, + "threshold": -3.018948793411255, + "operator": "ge", + "actual_fraction": 0.14, + "metric": 0.9766666666666667, + "rescued": 0, + "harmed": 15, + "missed_rescue": 2, + "avoided_harm": 513, + "selected_both_correct": 151, + "selected_both_wrong": 2 + }, + { + "budget": 0.15, + "threshold": -3.1879773139953613, + "operator": "ge", + "actual_fraction": 0.15, + "metric": 0.975, + "rescued": 0, + "harmed": 17, + "missed_rescue": 2, + "avoided_harm": 511, + "selected_both_correct": 160, + "selected_both_wrong": 3 + }, + { + "budget": 0.16, + "threshold": -3.447166919708252, + "operator": "ge", + "actual_fraction": 0.16, + "metric": 0.9741666666666666, + "rescued": 0, + "harmed": 18, + "missed_rescue": 2, + "avoided_harm": 510, + "selected_both_correct": 171, + "selected_both_wrong": 3 + }, + { + "budget": 0.17, + "threshold": -3.6728925704956055, + "operator": "ge", + "actual_fraction": 0.17, + "metric": 0.9725, + "rescued": 0, + "harmed": 20, + "missed_rescue": 2, + "avoided_harm": 508, + "selected_both_correct": 181, + "selected_both_wrong": 3 + }, + { + "budget": 0.18, + "threshold": -3.903853416442871, + "operator": "ge", + "actual_fraction": 0.18, + "metric": 0.9683333333333334, + "rescued": 0, + "harmed": 25, + "missed_rescue": 2, + "avoided_harm": 503, + "selected_both_correct": 188, + "selected_both_wrong": 3 + }, + { + "budget": 0.19, + "threshold": -4.163486480712891, + "operator": "ge", + "actual_fraction": 0.19, + "metric": 0.9666666666666667, + "rescued": 0, + "harmed": 27, + "missed_rescue": 2, + "avoided_harm": 501, + "selected_both_correct": 198, + "selected_both_wrong": 3 + }, + { + "budget": 0.2, + "threshold": -4.358063697814941, + "operator": "ge", + "actual_fraction": 0.2, + "metric": 0.965, + "rescued": 0, + "harmed": 29, + "missed_rescue": 2, + "avoided_harm": 499, + "selected_both_correct": 208, + "selected_both_wrong": 3 + }, + { + "budget": 0.21, + "threshold": -4.6970601081848145, + "operator": "ge", + "actual_fraction": 0.21, + "metric": 0.9608333333333333, + "rescued": 0, + "harmed": 34, + "missed_rescue": 2, + "avoided_harm": 494, + "selected_both_correct": 215, + "selected_both_wrong": 3 + }, + { + "budget": 0.22, + "threshold": -5.033747673034668, + "operator": "ge", + "actual_fraction": 0.22, + "metric": 0.9591666666666666, + "rescued": 0, + "harmed": 36, + "missed_rescue": 2, + "avoided_harm": 492, + "selected_both_correct": 225, + "selected_both_wrong": 3 + }, + { + "budget": 0.23, + "threshold": -5.244104862213135, + "operator": "ge", + "actual_fraction": 0.23, + "metric": 0.9583333333333334, + "rescued": 0, + "harmed": 37, + "missed_rescue": 2, + "avoided_harm": 491, + "selected_both_correct": 236, + "selected_both_wrong": 3 + }, + { + "budget": 0.24, + "threshold": -5.421384334564209, + "operator": "ge", + "actual_fraction": 0.24, + "metric": 0.9566666666666667, + "rescued": 0, + "harmed": 39, + "missed_rescue": 2, + "avoided_harm": 489, + "selected_both_correct": 245, + "selected_both_wrong": 4 + }, + { + "budget": 0.25, + "threshold": -5.576960563659668, + "operator": "ge", + "actual_fraction": 0.25, + "metric": 0.9533333333333334, + "rescued": 0, + "harmed": 43, + "missed_rescue": 2, + "avoided_harm": 485, + "selected_both_correct": 253, + "selected_both_wrong": 4 + }, + { + "budget": 0.26, + "threshold": -5.7489848136901855, + "operator": "ge", + "actual_fraction": 0.26, + "metric": 0.9508333333333333, + "rescued": 0, + "harmed": 46, + "missed_rescue": 2, + "avoided_harm": 482, + "selected_both_correct": 262, + "selected_both_wrong": 4 + }, + { + "budget": 0.27, + "threshold": -5.895847797393799, + "operator": "ge", + "actual_fraction": 0.27, + "metric": 0.9491666666666667, + "rescued": 0, + "harmed": 48, + "missed_rescue": 2, + "avoided_harm": 480, + "selected_both_correct": 272, + "selected_both_wrong": 4 + }, + { + "budget": 0.28, + "threshold": -6.221223831176758, + "operator": "ge", + "actual_fraction": 0.28, + "metric": 0.9491666666666667, + "rescued": 0, + "harmed": 48, + "missed_rescue": 2, + "avoided_harm": 480, + "selected_both_correct": 284, + "selected_both_wrong": 4 + }, + { + "budget": 0.29, + "threshold": -6.32234001159668, + "operator": "ge", + "actual_fraction": 0.29, + "metric": 0.9483333333333334, + "rescued": 0, + "harmed": 49, + "missed_rescue": 2, + "avoided_harm": 479, + "selected_both_correct": 295, + "selected_both_wrong": 4 + }, + { + "budget": 0.3, + "threshold": -6.438248157501221, + "operator": "ge", + "actual_fraction": 0.3, + "metric": 0.945, + "rescued": 0, + "harmed": 53, + "missed_rescue": 2, + "avoided_harm": 475, + "selected_both_correct": 303, + "selected_both_wrong": 4 + }, + { + "budget": 0.31, + "threshold": -6.564482688903809, + "operator": "ge", + "actual_fraction": 0.31, + "metric": 0.9391666666666667, + "rescued": 0, + "harmed": 60, + "missed_rescue": 2, + "avoided_harm": 468, + "selected_both_correct": 308, + "selected_both_wrong": 4 + }, + { + "budget": 0.32, + "threshold": -6.778532028198242, + "operator": "ge", + "actual_fraction": 0.32, + "metric": 0.9358333333333333, + "rescued": 0, + "harmed": 64, + "missed_rescue": 2, + "avoided_harm": 464, + "selected_both_correct": 316, + "selected_both_wrong": 4 + }, + { + "budget": 0.33, + "threshold": -7.0371198654174805, + "operator": "ge", + "actual_fraction": 0.33, + "metric": 0.9341666666666667, + "rescued": 1, + "harmed": 67, + "missed_rescue": 1, + "avoided_harm": 461, + "selected_both_correct": 324, + "selected_both_wrong": 4 + }, + { + "budget": 0.34, + "threshold": -7.208042621612549, + "operator": "ge", + "actual_fraction": 0.34, + "metric": 0.9316666666666666, + "rescued": 1, + "harmed": 70, + "missed_rescue": 1, + "avoided_harm": 458, + "selected_both_correct": 333, + "selected_both_wrong": 4 + }, + { + "budget": 0.35, + "threshold": -7.3958282470703125, + "operator": "ge", + "actual_fraction": 0.35, + "metric": 0.9291666666666667, + "rescued": 1, + "harmed": 73, + "missed_rescue": 1, + "avoided_harm": 455, + "selected_both_correct": 342, + "selected_both_wrong": 4 + }, + { + "budget": 0.36, + "threshold": -7.572646617889404, + "operator": "ge", + "actual_fraction": 0.36, + "metric": 0.9258333333333333, + "rescued": 1, + "harmed": 77, + "missed_rescue": 1, + "avoided_harm": 451, + "selected_both_correct": 349, + "selected_both_wrong": 5 + }, + { + "budget": 0.37, + "threshold": -7.778913974761963, + "operator": "ge", + "actual_fraction": 0.37, + "metric": 0.9225, + "rescued": 1, + "harmed": 81, + "missed_rescue": 1, + "avoided_harm": 447, + "selected_both_correct": 357, + "selected_both_wrong": 5 + }, + { + "budget": 0.38, + "threshold": -7.98846960067749, + "operator": "ge", + "actual_fraction": 0.38, + "metric": 0.9191666666666667, + "rescued": 1, + "harmed": 85, + "missed_rescue": 1, + "avoided_harm": 443, + "selected_both_correct": 365, + "selected_both_wrong": 5 + }, + { + "budget": 0.39, + "threshold": -8.223892211914062, + "operator": "ge", + "actual_fraction": 0.39, + "metric": 0.915, + "rescued": 1, + "harmed": 90, + "missed_rescue": 1, + "avoided_harm": 438, + "selected_both_correct": 372, + "selected_both_wrong": 5 + }, + { + "budget": 0.4, + "threshold": -8.36971378326416, + "operator": "ge", + "actual_fraction": 0.4, + "metric": 0.9133333333333333, + "rescued": 1, + "harmed": 92, + "missed_rescue": 1, + "avoided_harm": 436, + "selected_both_correct": 381, + "selected_both_wrong": 6 + }, + { + "budget": 0.41, + "threshold": -8.517688751220703, + "operator": "ge", + "actual_fraction": 0.4091666666666667, + "metric": 0.9108333333333334, + "rescued": 1, + "harmed": 95, + "missed_rescue": 1, + "avoided_harm": 433, + "selected_both_correct": 389, + "selected_both_wrong": 6 + }, + { + "budget": 0.42, + "threshold": -8.621047973632812, + "operator": "ge", + "actual_fraction": 0.42, + "metric": 0.9083333333333333, + "rescued": 1, + "harmed": 98, + "missed_rescue": 1, + "avoided_harm": 430, + "selected_both_correct": 399, + "selected_both_wrong": 6 + }, + { + "budget": 0.43, + "threshold": -8.791482925415039, + "operator": "ge", + "actual_fraction": 0.43, + "metric": 0.9066666666666666, + "rescued": 1, + "harmed": 100, + "missed_rescue": 1, + "avoided_harm": 428, + "selected_both_correct": 409, + "selected_both_wrong": 6 + }, + { + "budget": 0.44, + "threshold": -8.940971374511719, + "operator": "ge", + "actual_fraction": 0.44, + "metric": 0.9016666666666666, + "rescued": 1, + "harmed": 106, + "missed_rescue": 1, + "avoided_harm": 422, + "selected_both_correct": 415, + "selected_both_wrong": 6 + }, + { + "budget": 0.45, + "threshold": -9.110553741455078, + "operator": "ge", + "actual_fraction": 0.45, + "metric": 0.8991666666666667, + "rescued": 1, + "harmed": 109, + "missed_rescue": 1, + "avoided_harm": 419, + "selected_both_correct": 424, + "selected_both_wrong": 6 + }, + { + "budget": 0.46, + "threshold": -9.255155563354492, + "operator": "ge", + "actual_fraction": 0.46, + "metric": 0.8958333333333334, + "rescued": 1, + "harmed": 113, + "missed_rescue": 1, + "avoided_harm": 415, + "selected_both_correct": 432, + "selected_both_wrong": 6 + }, + { + "budget": 0.47, + "threshold": -9.383505821228027, + "operator": "ge", + "actual_fraction": 0.47, + "metric": 0.8925, + "rescued": 1, + "harmed": 117, + "missed_rescue": 1, + "avoided_harm": 411, + "selected_both_correct": 440, + "selected_both_wrong": 6 + }, + { + "budget": 0.48, + "threshold": -9.505346298217773, + "operator": "ge", + "actual_fraction": 0.48, + "metric": 0.8883333333333333, + "rescued": 1, + "harmed": 122, + "missed_rescue": 1, + "avoided_harm": 406, + "selected_both_correct": 446, + "selected_both_wrong": 7 + }, + { + "budget": 0.49, + "threshold": -9.638259887695312, + "operator": "ge", + "actual_fraction": 0.49, + "metric": 0.885, + "rescued": 1, + "harmed": 126, + "missed_rescue": 1, + "avoided_harm": 402, + "selected_both_correct": 454, + "selected_both_wrong": 7 + }, + { + "budget": 0.5, + "threshold": -9.84414291381836, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.88, + "rescued": 1, + "harmed": 132, + "missed_rescue": 1, + "avoided_harm": 396, + "selected_both_correct": 460, + "selected_both_wrong": 7 + }, + { + "budget": 0.51, + "threshold": -10.044600486755371, + "operator": "ge", + "actual_fraction": 0.51, + "metric": 0.8791666666666667, + "rescued": 1, + "harmed": 133, + "missed_rescue": 1, + "avoided_harm": 395, + "selected_both_correct": 471, + "selected_both_wrong": 7 + }, + { + "budget": 0.52, + "threshold": -10.237942695617676, + "operator": "ge", + "actual_fraction": 0.52, + "metric": 0.8741666666666666, + "rescued": 1, + "harmed": 139, + "missed_rescue": 1, + "avoided_harm": 389, + "selected_both_correct": 477, + "selected_both_wrong": 7 + }, + { + "budget": 0.53, + "threshold": -10.36861801147461, + "operator": "ge", + "actual_fraction": 0.53, + "metric": 0.8691666666666666, + "rescued": 1, + "harmed": 145, + "missed_rescue": 1, + "avoided_harm": 383, + "selected_both_correct": 483, + "selected_both_wrong": 7 + }, + { + "budget": 0.54, + "threshold": -10.604632377624512, + "operator": "ge", + "actual_fraction": 0.54, + "metric": 0.865, + "rescued": 1, + "harmed": 150, + "missed_rescue": 1, + "avoided_harm": 378, + "selected_both_correct": 489, + "selected_both_wrong": 8 + }, + { + "budget": 0.55, + "threshold": -10.71943473815918, + "operator": "ge", + "actual_fraction": 0.55, + "metric": 0.8591666666666666, + "rescued": 1, + "harmed": 157, + "missed_rescue": 1, + "avoided_harm": 371, + "selected_both_correct": 494, + "selected_both_wrong": 8 + }, + { + "budget": 0.56, + "threshold": -10.831548690795898, + "operator": "ge", + "actual_fraction": 0.56, + "metric": 0.855, + "rescued": 1, + "harmed": 162, + "missed_rescue": 1, + "avoided_harm": 366, + "selected_both_correct": 501, + "selected_both_wrong": 8 + }, + { + "budget": 0.57, + "threshold": -11.046300888061523, + "operator": "ge", + "actual_fraction": 0.5691666666666667, + "metric": 0.8491666666666666, + "rescued": 1, + "harmed": 169, + "missed_rescue": 1, + "avoided_harm": 359, + "selected_both_correct": 505, + "selected_both_wrong": 8 + }, + { + "budget": 0.58, + "threshold": -11.17777156829834, + "operator": "ge", + "actual_fraction": 0.58, + "metric": 0.8458333333333333, + "rescued": 1, + "harmed": 173, + "missed_rescue": 1, + "avoided_harm": 355, + "selected_both_correct": 514, + "selected_both_wrong": 8 + }, + { + "budget": 0.59, + "threshold": -11.355005264282227, + "operator": "ge", + "actual_fraction": 0.59, + "metric": 0.84, + "rescued": 1, + "harmed": 180, + "missed_rescue": 1, + "avoided_harm": 348, + "selected_both_correct": 519, + "selected_both_wrong": 8 + }, + { + "budget": 0.6, + "threshold": -11.538931846618652, + "operator": "ge", + "actual_fraction": 0.6, + "metric": 0.835, + "rescued": 1, + "harmed": 186, + "missed_rescue": 1, + "avoided_harm": 342, + "selected_both_correct": 525, + "selected_both_wrong": 8 + }, + { + "budget": 0.61, + "threshold": -11.703760147094727, + "operator": "ge", + "actual_fraction": 0.61, + "metric": 0.8308333333333333, + "rescued": 1, + "harmed": 191, + "missed_rescue": 1, + "avoided_harm": 337, + "selected_both_correct": 532, + "selected_both_wrong": 8 + }, + { + "budget": 0.62, + "threshold": -11.831314086914062, + "operator": "ge", + "actual_fraction": 0.62, + "metric": 0.8233333333333334, + "rescued": 1, + "harmed": 200, + "missed_rescue": 1, + "avoided_harm": 328, + "selected_both_correct": 535, + "selected_both_wrong": 8 + }, + { + "budget": 0.63, + "threshold": -12.021162986755371, + "operator": "ge", + "actual_fraction": 0.63, + "metric": 0.8175, + "rescued": 1, + "harmed": 207, + "missed_rescue": 1, + "avoided_harm": 321, + "selected_both_correct": 540, + "selected_both_wrong": 8 + }, + { + "budget": 0.64, + "threshold": -12.189552307128906, + "operator": "ge", + "actual_fraction": 0.64, + "metric": 0.8116666666666666, + "rescued": 1, + "harmed": 214, + "missed_rescue": 1, + "avoided_harm": 314, + "selected_both_correct": 545, + "selected_both_wrong": 8 + }, + { + "budget": 0.65, + "threshold": -12.395832061767578, + "operator": "ge", + "actual_fraction": 0.65, + "metric": 0.805, + "rescued": 1, + "harmed": 222, + "missed_rescue": 1, + "avoided_harm": 306, + "selected_both_correct": 549, + "selected_both_wrong": 8 + }, + { + "budget": 0.66, + "threshold": -12.56098461151123, + "operator": "ge", + "actual_fraction": 0.66, + "metric": 0.8016666666666666, + "rescued": 1, + "harmed": 226, + "missed_rescue": 1, + "avoided_harm": 302, + "selected_both_correct": 557, + "selected_both_wrong": 8 + }, + { + "budget": 0.67, + "threshold": -12.731056213378906, + "operator": "ge", + "actual_fraction": 0.67, + "metric": 0.7958333333333333, + "rescued": 1, + "harmed": 233, + "missed_rescue": 1, + "avoided_harm": 295, + "selected_both_correct": 562, + "selected_both_wrong": 8 + }, + { + "budget": 0.68, + "threshold": -12.946990966796875, + "operator": "ge", + "actual_fraction": 0.68, + "metric": 0.7916666666666666, + "rescued": 1, + "harmed": 238, + "missed_rescue": 1, + "avoided_harm": 290, + "selected_both_correct": 569, + "selected_both_wrong": 8 + }, + { + "budget": 0.69, + "threshold": -13.060778617858887, + "operator": "ge", + "actual_fraction": 0.6891666666666667, + "metric": 0.7858333333333334, + "rescued": 1, + "harmed": 245, + "missed_rescue": 1, + "avoided_harm": 283, + "selected_both_correct": 573, + "selected_both_wrong": 8 + }, + { + "budget": 0.7, + "threshold": -13.237710952758789, + "operator": "ge", + "actual_fraction": 0.7, + "metric": 0.7791666666666667, + "rescued": 1, + "harmed": 253, + "missed_rescue": 1, + "avoided_harm": 275, + "selected_both_correct": 578, + "selected_both_wrong": 8 + }, + { + "budget": 0.71, + "threshold": -13.452357292175293, + "operator": "ge", + "actual_fraction": 0.71, + "metric": 0.775, + "rescued": 1, + "harmed": 258, + "missed_rescue": 1, + "avoided_harm": 270, + "selected_both_correct": 585, + "selected_both_wrong": 8 + }, + { + "budget": 0.72, + "threshold": -13.714129447937012, + "operator": "ge", + "actual_fraction": 0.72, + "metric": 0.7708333333333334, + "rescued": 1, + "harmed": 263, + "missed_rescue": 1, + "avoided_harm": 265, + "selected_both_correct": 592, + "selected_both_wrong": 8 + }, + { + "budget": 0.73, + "threshold": -13.887096405029297, + "operator": "ge", + "actual_fraction": 0.73, + "metric": 0.7641666666666667, + "rescued": 1, + "harmed": 271, + "missed_rescue": 1, + "avoided_harm": 257, + "selected_both_correct": 596, + "selected_both_wrong": 8 + }, + { + "budget": 0.74, + "threshold": -14.114652633666992, + "operator": "ge", + "actual_fraction": 0.74, + "metric": 0.7583333333333333, + "rescued": 1, + "harmed": 278, + "missed_rescue": 1, + "avoided_harm": 250, + "selected_both_correct": 601, + "selected_both_wrong": 8 + }, + { + "budget": 0.75, + "threshold": -14.356359481811523, + "operator": "ge", + "actual_fraction": 0.75, + "metric": 0.7516666666666667, + "rescued": 1, + "harmed": 286, + "missed_rescue": 1, + "avoided_harm": 242, + "selected_both_correct": 605, + "selected_both_wrong": 8 + }, + { + "budget": 0.76, + "threshold": -14.591072082519531, + "operator": "ge", + "actual_fraction": 0.76, + "metric": 0.7475, + "rescued": 2, + "harmed": 292, + "missed_rescue": 0, + "avoided_harm": 236, + "selected_both_correct": 610, + "selected_both_wrong": 8 + }, + { + "budget": 0.77, + "threshold": -15.056472778320312, + "operator": "ge", + "actual_fraction": 0.77, + "metric": 0.7425, + "rescued": 2, + "harmed": 298, + "missed_rescue": 0, + "avoided_harm": 230, + "selected_both_correct": 615, + "selected_both_wrong": 9 + }, + { + "budget": 0.78, + "threshold": -15.384273529052734, + "operator": "ge", + "actual_fraction": 0.78, + "metric": 0.7333333333333333, + "rescued": 2, + "harmed": 309, + "missed_rescue": 0, + "avoided_harm": 219, + "selected_both_correct": 616, + "selected_both_wrong": 9 + }, + { + "budget": 0.79, + "threshold": -15.68124008178711, + "operator": "ge", + "actual_fraction": 0.79, + "metric": 0.7233333333333334, + "rescued": 2, + "harmed": 321, + "missed_rescue": 0, + "avoided_harm": 207, + "selected_both_correct": 616, + "selected_both_wrong": 9 + }, + { + "budget": 0.8, + "threshold": -15.871927261352539, + "operator": "ge", + "actual_fraction": 0.8, + "metric": 0.7158333333333333, + "rescued": 2, + "harmed": 330, + "missed_rescue": 0, + "avoided_harm": 198, + "selected_both_correct": 619, + "selected_both_wrong": 9 + }, + { + "budget": 0.81, + "threshold": -16.09673500061035, + "operator": "ge", + "actual_fraction": 0.81, + "metric": 0.71, + "rescued": 2, + "harmed": 337, + "missed_rescue": 0, + "avoided_harm": 191, + "selected_both_correct": 624, + "selected_both_wrong": 9 + }, + { + "budget": 0.82, + "threshold": -16.361101150512695, + "operator": "ge", + "actual_fraction": 0.8191666666666667, + "metric": 0.705, + "rescued": 2, + "harmed": 343, + "missed_rescue": 0, + "avoided_harm": 185, + "selected_both_correct": 629, + "selected_both_wrong": 9 + }, + { + "budget": 0.83, + "threshold": -16.775108337402344, + "operator": "ge", + "actual_fraction": 0.83, + "metric": 0.6983333333333334, + "rescued": 2, + "harmed": 351, + "missed_rescue": 0, + "avoided_harm": 177, + "selected_both_correct": 634, + "selected_both_wrong": 9 + }, + { + "budget": 0.84, + "threshold": -17.00735855102539, + "operator": "ge", + "actual_fraction": 0.84, + "metric": 0.6908333333333333, + "rescued": 2, + "harmed": 360, + "missed_rescue": 0, + "avoided_harm": 168, + "selected_both_correct": 637, + "selected_both_wrong": 9 + }, + { + "budget": 0.85, + "threshold": -17.19187355041504, + "operator": "ge", + "actual_fraction": 0.85, + "metric": 0.6825, + "rescued": 2, + "harmed": 370, + "missed_rescue": 0, + "avoided_harm": 158, + "selected_both_correct": 639, + "selected_both_wrong": 9 + }, + { + "budget": 0.86, + "threshold": -17.789541244506836, + "operator": "ge", + "actual_fraction": 0.86, + "metric": 0.675, + "rescued": 2, + "harmed": 379, + "missed_rescue": 0, + "avoided_harm": 149, + "selected_both_correct": 642, + "selected_both_wrong": 9 + }, + { + "budget": 0.87, + "threshold": -18.021289825439453, + "operator": "ge", + "actual_fraction": 0.87, + "metric": 0.6683333333333333, + "rescued": 2, + "harmed": 387, + "missed_rescue": 0, + "avoided_harm": 141, + "selected_both_correct": 646, + "selected_both_wrong": 9 + }, + { + "budget": 0.88, + "threshold": -18.256092071533203, + "operator": "ge", + "actual_fraction": 0.88, + "metric": 0.6625, + "rescued": 2, + "harmed": 394, + "missed_rescue": 0, + "avoided_harm": 134, + "selected_both_correct": 650, + "selected_both_wrong": 10 + }, + { + "budget": 0.89, + "threshold": -18.753929138183594, + "operator": "ge", + "actual_fraction": 0.89, + "metric": 0.6541666666666667, + "rescued": 2, + "harmed": 404, + "missed_rescue": 0, + "avoided_harm": 124, + "selected_both_correct": 652, + "selected_both_wrong": 10 + }, + { + "budget": 0.9, + "threshold": -19.046903610229492, + "operator": "ge", + "actual_fraction": 0.9, + "metric": 0.6441666666666667, + "rescued": 2, + "harmed": 416, + "missed_rescue": 0, + "avoided_harm": 112, + "selected_both_correct": 652, + "selected_both_wrong": 10 + }, + { + "budget": 0.91, + "threshold": -19.4178466796875, + "operator": "ge", + "actual_fraction": 0.91, + "metric": 0.6341666666666667, + "rescued": 2, + "harmed": 428, + "missed_rescue": 0, + "avoided_harm": 100, + "selected_both_correct": 652, + "selected_both_wrong": 10 + }, + { + "budget": 0.92, + "threshold": -19.944700241088867, + "operator": "ge", + "actual_fraction": 0.92, + "metric": 0.6258333333333334, + "rescued": 2, + "harmed": 438, + "missed_rescue": 0, + "avoided_harm": 90, + "selected_both_correct": 654, + "selected_both_wrong": 10 + }, + { + "budget": 0.93, + "threshold": -20.247802734375, + "operator": "ge", + "actual_fraction": 0.93, + "metric": 0.6166666666666667, + "rescued": 2, + "harmed": 449, + "missed_rescue": 0, + "avoided_harm": 79, + "selected_both_correct": 655, + "selected_both_wrong": 10 + }, + { + "budget": 0.94, + "threshold": -21.42934226989746, + "operator": "ge", + "actual_fraction": 0.94, + "metric": 0.6075, + "rescued": 2, + "harmed": 460, + "missed_rescue": 0, + "avoided_harm": 68, + "selected_both_correct": 656, + "selected_both_wrong": 10 + }, + { + "budget": 0.95, + "threshold": -22.04431915283203, + "operator": "ge", + "actual_fraction": 0.95, + "metric": 0.5983333333333334, + "rescued": 2, + "harmed": 471, + "missed_rescue": 0, + "avoided_harm": 57, + "selected_both_correct": 657, + "selected_both_wrong": 10 + }, + { + "budget": 0.96, + "threshold": -23.509872436523438, + "operator": "ge", + "actual_fraction": 0.96, + "metric": 0.5891666666666666, + "rescued": 2, + "harmed": 482, + "missed_rescue": 0, + "avoided_harm": 46, + "selected_both_correct": 658, + "selected_both_wrong": 10 + }, + { + "budget": 0.97, + "threshold": -24.155170440673828, + "operator": "ge", + "actual_fraction": 0.97, + "metric": 0.5791666666666667, + "rescued": 2, + "harmed": 494, + "missed_rescue": 0, + "avoided_harm": 34, + "selected_both_correct": 658, + "selected_both_wrong": 10 + }, + { + "budget": 0.98, + "threshold": -25.53754234313965, + "operator": "ge", + "actual_fraction": 0.98, + "metric": 0.57, + "rescued": 2, + "harmed": 505, + "missed_rescue": 0, + "avoided_harm": 23, + "selected_both_correct": 658, + "selected_both_wrong": 11 + }, + { + "budget": 0.99, + "threshold": -27.78575325012207, + "operator": "ge", + "actual_fraction": 0.99, + "metric": 0.5608333333333333, + "rescued": 2, + "harmed": 516, + "missed_rescue": 0, + "avoided_harm": 12, + "selected_both_correct": 659, + "selected_both_wrong": 11 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5508333333333333, + "rescued": 2, + "harmed": 528, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 659, + "selected_both_wrong": 11 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9016223679668622, + "rescued": 0, + "harmed": 0, + "missed_rescue": 144, + "avoided_harm": 2192, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 4.029286861419678, + "operator": "ge", + "actual_fraction": 0.016568864342423197, + "metric": 0.9005868139454608, + "rescued": 0, + "harmed": 6, + "missed_rescue": 144, + "avoided_harm": 2186, + "selected_both_correct": 86, + "selected_both_wrong": 4 + }, + { + "budget": 0.02, + "threshold": 2.9611759185791016, + "operator": "ge", + "actual_fraction": 0.025716258198136003, + "metric": 0.9009319986192613, + "rescued": 3, + "harmed": 7, + "missed_rescue": 141, + "avoided_harm": 2185, + "selected_both_correct": 132, + "selected_both_wrong": 7 + }, + { + "budget": 0.03, + "threshold": 1.9980177879333496, + "operator": "ge", + "actual_fraction": 0.03710735243355195, + "metric": 0.9004142216085605, + "rescued": 3, + "harmed": 10, + "missed_rescue": 141, + "avoided_harm": 2182, + "selected_both_correct": 193, + "selected_both_wrong": 9 + }, + { + "budget": 0.04, + "threshold": 1.3639881610870361, + "operator": "ge", + "actual_fraction": 0.04763548498446669, + "metric": 0.8998964445978599, + "rescued": 3, + "harmed": 13, + "missed_rescue": 141, + "avoided_harm": 2179, + "selected_both_correct": 246, + "selected_both_wrong": 14 + }, + { + "budget": 0.05, + "threshold": 0.9321551322937012, + "operator": "ge", + "actual_fraction": 0.05350362443907491, + "metric": 0.8995512599240594, + "rescued": 3, + "harmed": 15, + "missed_rescue": 141, + "avoided_harm": 2177, + "selected_both_correct": 278, + "selected_both_wrong": 14 + }, + { + "budget": 0.06, + "threshold": 0.3624706268310547, + "operator": "ge", + "actual_fraction": 0.06351397997928893, + "metric": 0.8992060752502589, + "rescued": 5, + "harmed": 19, + "missed_rescue": 139, + "avoided_harm": 2173, + "selected_both_correct": 326, + "selected_both_wrong": 18 + }, + { + "budget": 0.07, + "threshold": -0.07812309265136719, + "operator": "ge", + "actual_fraction": 0.0762858129099068, + "metric": 0.8978253365550569, + "rescued": 6, + "harmed": 28, + "missed_rescue": 138, + "avoided_harm": 2164, + "selected_both_correct": 386, + "selected_both_wrong": 22 + }, + { + "budget": 0.08, + "threshold": -0.7457926273345947, + "operator": "ge", + "actual_fraction": 0.09112875388332758, + "metric": 0.897134967207456, + "rescued": 9, + "harmed": 35, + "missed_rescue": 135, + "avoided_harm": 2157, + "selected_both_correct": 457, + "selected_both_wrong": 27 + }, + { + "budget": 0.09, + "threshold": -1.2083253860473633, + "operator": "ge", + "actual_fraction": 0.10424577148774594, + "metric": 0.8962720055229548, + "rescued": 12, + "harmed": 43, + "missed_rescue": 132, + "avoided_harm": 2149, + "selected_both_correct": 513, + "selected_both_wrong": 36 + }, + { + "budget": 0.1, + "threshold": -1.6223034858703613, + "operator": "ge", + "actual_fraction": 0.1164998274076631, + "metric": 0.8954090438384535, + "rescued": 13, + "harmed": 49, + "missed_rescue": 131, + "avoided_harm": 2143, + "selected_both_correct": 577, + "selected_both_wrong": 36 + }, + { + "budget": 0.11, + "threshold": -2.024738073348999, + "operator": "ge", + "actual_fraction": 0.12702795995857785, + "metric": 0.8947186744908526, + "rescued": 14, + "harmed": 54, + "missed_rescue": 130, + "avoided_harm": 2138, + "selected_both_correct": 628, + "selected_both_wrong": 40 + }, + { + "budget": 0.12, + "threshold": -2.3822429180145264, + "operator": "ge", + "actual_fraction": 0.1380738695201933, + "metric": 0.8950638591646531, + "rescued": 18, + "harmed": 56, + "missed_rescue": 126, + "avoided_harm": 2136, + "selected_both_correct": 680, + "selected_both_wrong": 46 + }, + { + "budget": 0.13, + "threshold": -2.7217001914978027, + "operator": "ge", + "actual_fraction": 0.14929237141870902, + "metric": 0.8940283051432516, + "rescued": 19, + "harmed": 63, + "missed_rescue": 125, + "avoided_harm": 2129, + "selected_both_correct": 735, + "selected_both_wrong": 48 + }, + { + "budget": 0.14, + "threshold": -3.018948793411255, + "operator": "ge", + "actual_fraction": 0.15982050396962375, + "metric": 0.8923023817742493, + "rescued": 19, + "harmed": 73, + "missed_rescue": 125, + "avoided_harm": 2119, + "selected_both_correct": 783, + "selected_both_wrong": 51 + }, + { + "budget": 0.15, + "threshold": -3.1879773139953613, + "operator": "ge", + "actual_fraction": 0.16637901277183292, + "metric": 0.8917846047635485, + "rescued": 20, + "harmed": 77, + "missed_rescue": 124, + "avoided_harm": 2115, + "selected_both_correct": 812, + "selected_both_wrong": 55 + }, + { + "budget": 0.16, + "threshold": -3.447166919708252, + "operator": "ge", + "actual_fraction": 0.17604418363824645, + "metric": 0.8904038660683465, + "rescued": 20, + "harmed": 85, + "missed_rescue": 124, + "avoided_harm": 2107, + "selected_both_correct": 854, + "selected_both_wrong": 61 + }, + { + "budget": 0.17, + "threshold": -3.6728925704956055, + "operator": "ge", + "actual_fraction": 0.18622713151536072, + "metric": 0.8898860890576459, + "rescued": 23, + "harmed": 91, + "missed_rescue": 121, + "avoided_harm": 2101, + "selected_both_correct": 899, + "selected_both_wrong": 66 + }, + { + "budget": 0.18, + "threshold": -3.903853416442871, + "operator": "ge", + "actual_fraction": 0.19589230238177424, + "metric": 0.8878149810148429, + "rescued": 24, + "harmed": 104, + "missed_rescue": 120, + "avoided_harm": 2088, + "selected_both_correct": 937, + "selected_both_wrong": 70 + }, + { + "budget": 0.19, + "threshold": -4.163486480712891, + "operator": "ge", + "actual_fraction": 0.20883672764929237, + "metric": 0.886434242319641, + "rescued": 24, + "harmed": 112, + "missed_rescue": 120, + "avoided_harm": 2080, + "selected_both_correct": 999, + "selected_both_wrong": 75 + }, + { + "budget": 0.2, + "threshold": -4.358063697814941, + "operator": "ge", + "actual_fraction": 0.21712115982050398, + "metric": 0.8845357266137384, + "rescued": 24, + "harmed": 123, + "missed_rescue": 120, + "avoided_harm": 2069, + "selected_both_correct": 1032, + "selected_both_wrong": 79 + }, + { + "budget": 0.21, + "threshold": -4.6970601081848145, + "operator": "ge", + "actual_fraction": 0.2317915084570245, + "metric": 0.8812564722126338, + "rescued": 25, + "harmed": 143, + "missed_rescue": 119, + "avoided_harm": 2049, + "selected_both_correct": 1089, + "selected_both_wrong": 86 + }, + { + "budget": 0.22, + "threshold": -5.033747673034668, + "operator": "ge", + "actual_fraction": 0.2462892647566448, + "metric": 0.8790127718329306, + "rescued": 26, + "harmed": 157, + "missed_rescue": 118, + "avoided_harm": 2035, + "selected_both_correct": 1151, + "selected_both_wrong": 93 + }, + { + "budget": 0.23, + "threshold": -5.244104862213135, + "operator": "ge", + "actual_fraction": 0.2554366586123576, + "metric": 0.8767690714532275, + "rescued": 27, + "harmed": 171, + "missed_rescue": 117, + "avoided_harm": 2021, + "selected_both_correct": 1187, + "selected_both_wrong": 95 + }, + { + "budget": 0.24, + "threshold": -5.421384334564209, + "operator": "ge", + "actual_fraction": 0.26389368312046946, + "metric": 0.8755609250949258, + "rescued": 27, + "harmed": 178, + "missed_rescue": 117, + "avoided_harm": 2014, + "selected_both_correct": 1225, + "selected_both_wrong": 99 + }, + { + "budget": 0.25, + "threshold": -5.576960563659668, + "operator": "ge", + "actual_fraction": 0.2697618225750777, + "metric": 0.8743527787366241, + "rescued": 28, + "harmed": 186, + "missed_rescue": 116, + "avoided_harm": 2006, + "selected_both_correct": 1247, + "selected_both_wrong": 102 + }, + { + "budget": 0.26, + "threshold": -5.7489848136901855, + "operator": "ge", + "actual_fraction": 0.2806351397997929, + "metric": 0.8729720400414221, + "rescued": 33, + "harmed": 199, + "missed_rescue": 111, + "avoided_harm": 1993, + "selected_both_correct": 1289, + "selected_both_wrong": 105 + }, + { + "budget": 0.27, + "threshold": -5.895847797393799, + "operator": "ge", + "actual_fraction": 0.28736624093890234, + "metric": 0.8712461166724197, + "rescued": 35, + "harmed": 211, + "missed_rescue": 109, + "avoided_harm": 1981, + "selected_both_correct": 1312, + "selected_both_wrong": 107 + }, + { + "budget": 0.28, + "threshold": -6.221223831176758, + "operator": "ge", + "actual_fraction": 0.30376251294442524, + "metric": 0.8676216775975146, + "rescued": 36, + "harmed": 233, + "missed_rescue": 108, + "avoided_harm": 1959, + "selected_both_correct": 1377, + "selected_both_wrong": 114 + }, + { + "budget": 0.29, + "threshold": -6.32234001159668, + "operator": "ge", + "actual_fraction": 0.30911287538833276, + "metric": 0.8665861235761132, + "rescued": 37, + "harmed": 240, + "missed_rescue": 107, + "avoided_harm": 1952, + "selected_both_correct": 1397, + "selected_both_wrong": 117 + }, + { + "budget": 0.3, + "threshold": -6.438248157501221, + "operator": "ge", + "actual_fraction": 0.3139454608215395, + "metric": 0.8655505695547118, + "rescued": 37, + "harmed": 246, + "missed_rescue": 107, + "avoided_harm": 1946, + "selected_both_correct": 1418, + "selected_both_wrong": 118 + }, + { + "budget": 0.31, + "threshold": -6.564482688903809, + "operator": "ge", + "actual_fraction": 0.319986192613048, + "metric": 0.8639972385226096, + "rescued": 39, + "harmed": 257, + "missed_rescue": 105, + "avoided_harm": 1935, + "selected_both_correct": 1438, + "selected_both_wrong": 120 + }, + { + "budget": 0.32, + "threshold": -6.778532028198242, + "operator": "ge", + "actual_fraction": 0.33103210217466345, + "metric": 0.8614083534691059, + "rescued": 41, + "harmed": 274, + "missed_rescue": 103, + "avoided_harm": 1918, + "selected_both_correct": 1480, + "selected_both_wrong": 123 + }, + { + "budget": 0.33, + "threshold": -7.0371198654174805, + "operator": "ge", + "actual_fraction": 0.34673800483258543, + "metric": 0.8574387297204004, + "rescued": 43, + "harmed": 299, + "missed_rescue": 101, + "avoided_harm": 1893, + "selected_both_correct": 1540, + "selected_both_wrong": 127 + }, + { + "budget": 0.34, + "threshold": -7.208042621612549, + "operator": "ge", + "actual_fraction": 0.3560579910251985, + "metric": 0.855712806351398, + "rescued": 43, + "harmed": 309, + "missed_rescue": 101, + "avoided_harm": 1883, + "selected_both_correct": 1583, + "selected_both_wrong": 128 + }, + { + "budget": 0.35, + "threshold": -7.3958282470703125, + "operator": "ge", + "actual_fraction": 0.36727649292371417, + "metric": 0.8534691059716949, + "rescued": 44, + "harmed": 323, + "missed_rescue": 100, + "avoided_harm": 1869, + "selected_both_correct": 1633, + "selected_both_wrong": 128 + }, + { + "budget": 0.36, + "threshold": -7.572646617889404, + "operator": "ge", + "actual_fraction": 0.37452537107352435, + "metric": 0.8519157749395927, + "rescued": 44, + "harmed": 332, + "missed_rescue": 100, + "avoided_harm": 1860, + "selected_both_correct": 1662, + "selected_both_wrong": 132 + }, + { + "budget": 0.37, + "threshold": -7.778913974761963, + "operator": "ge", + "actual_fraction": 0.38695201933034173, + "metric": 0.8477735588539869, + "rescued": 44, + "harmed": 356, + "missed_rescue": 100, + "avoided_harm": 1836, + "selected_both_correct": 1702, + "selected_both_wrong": 140 + }, + { + "budget": 0.38, + "threshold": -7.98846960067749, + "operator": "ge", + "actual_fraction": 0.397825336555057, + "metric": 0.845702450811184, + "rescued": 46, + "harmed": 370, + "missed_rescue": 98, + "avoided_harm": 1822, + "selected_both_correct": 1741, + "selected_both_wrong": 148 + }, + { + "budget": 0.39, + "threshold": -8.223892211914062, + "operator": "ge", + "actual_fraction": 0.41266827752847773, + "metric": 0.8427683810838799, + "rescued": 48, + "harmed": 389, + "missed_rescue": 96, + "avoided_harm": 1803, + "selected_both_correct": 1800, + "selected_both_wrong": 154 + }, + { + "budget": 0.4, + "threshold": -8.36971378326416, + "operator": "ge", + "actual_fraction": 0.4204349326889886, + "metric": 0.8405246807041767, + "rescued": 50, + "harmed": 404, + "missed_rescue": 94, + "avoided_harm": 1788, + "selected_both_correct": 1827, + "selected_both_wrong": 155 + }, + { + "budget": 0.41, + "threshold": -8.517688751220703, + "operator": "ge", + "actual_fraction": 0.42889195719710044, + "metric": 0.8377632033137729, + "rescued": 52, + "harmed": 422, + "missed_rescue": 92, + "avoided_harm": 1770, + "selected_both_correct": 1853, + "selected_both_wrong": 158 + }, + { + "budget": 0.42, + "threshold": -8.621047973632812, + "operator": "ge", + "actual_fraction": 0.43424231964100796, + "metric": 0.8362098722816707, + "rescued": 52, + "harmed": 431, + "missed_rescue": 92, + "avoided_harm": 1761, + "selected_both_correct": 1872, + "selected_both_wrong": 161 + }, + { + "budget": 0.43, + "threshold": -8.791482925415039, + "operator": "ge", + "actual_fraction": 0.4432171211598205, + "metric": 0.8331032102174664, + "rescued": 52, + "harmed": 449, + "missed_rescue": 92, + "avoided_harm": 1743, + "selected_both_correct": 1897, + "selected_both_wrong": 170 + }, + { + "budget": 0.44, + "threshold": -8.940971374511719, + "operator": "ge", + "actual_fraction": 0.45460821539523644, + "metric": 0.8287884017949603, + "rescued": 54, + "harmed": 476, + "missed_rescue": 90, + "avoided_harm": 1716, + "selected_both_correct": 1930, + "selected_both_wrong": 174 + }, + { + "budget": 0.45, + "threshold": -9.110553741455078, + "operator": "ge", + "actual_fraction": 0.4654815326199517, + "metric": 0.825681739730756, + "rescued": 55, + "harmed": 495, + "missed_rescue": 89, + "avoided_harm": 1697, + "selected_both_correct": 1970, + "selected_both_wrong": 177 + }, + { + "budget": 0.46, + "threshold": -9.255155563354492, + "operator": "ge", + "actual_fraction": 0.4770452191922679, + "metric": 0.8225750776665516, + "rescued": 57, + "harmed": 515, + "missed_rescue": 87, + "avoided_harm": 1677, + "selected_both_correct": 2008, + "selected_both_wrong": 184 + }, + { + "budget": 0.47, + "threshold": -9.383505821228027, + "operator": "ge", + "actual_fraction": 0.4855022437003797, + "metric": 0.8196410079392475, + "rescued": 57, + "harmed": 532, + "missed_rescue": 87, + "avoided_harm": 1660, + "selected_both_correct": 2037, + "selected_both_wrong": 187 + }, + { + "budget": 0.48, + "threshold": -9.505346298217773, + "operator": "ge", + "actual_fraction": 0.4932688988608906, + "metric": 0.8173973075595443, + "rescued": 57, + "harmed": 545, + "missed_rescue": 87, + "avoided_harm": 1647, + "selected_both_correct": 2064, + "selected_both_wrong": 192 + }, + { + "budget": 0.49, + "threshold": -9.638259887695312, + "operator": "ge", + "actual_fraction": 0.5013807386952019, + "metric": 0.8148084225060407, + "rescued": 59, + "harmed": 562, + "missed_rescue": 85, + "avoided_harm": 1630, + "selected_both_correct": 2091, + "selected_both_wrong": 193 + }, + { + "budget": 0.5, + "threshold": -9.84414291381836, + "operator": "ge", + "actual_fraction": 0.5120814635830169, + "metric": 0.810666206420435, + "rescued": 60, + "harmed": 587, + "missed_rescue": 84, + "avoided_harm": 1605, + "selected_both_correct": 2123, + "selected_both_wrong": 197 + }, + { + "budget": 0.51, + "threshold": -10.044600486755371, + "operator": "ge", + "actual_fraction": 0.5267518122195375, + "metric": 0.805661028650328, + "rescued": 62, + "harmed": 618, + "missed_rescue": 82, + "avoided_harm": 1574, + "selected_both_correct": 2170, + "selected_both_wrong": 202 + }, + { + "budget": 0.52, + "threshold": -10.237942695617676, + "operator": "ge", + "actual_fraction": 0.5374525371073524, + "metric": 0.8016914049016224, + "rescued": 63, + "harmed": 642, + "missed_rescue": 81, + "avoided_harm": 1550, + "selected_both_correct": 2206, + "selected_both_wrong": 203 + }, + { + "budget": 0.53, + "threshold": -10.36861801147461, + "operator": "ge", + "actual_fraction": 0.5421125302036589, + "metric": 0.7994477045219193, + "rescued": 63, + "harmed": 655, + "missed_rescue": 81, + "avoided_harm": 1537, + "selected_both_correct": 2220, + "selected_both_wrong": 203 + }, + { + "budget": 0.54, + "threshold": -10.604632377624512, + "operator": "ge", + "actual_fraction": 0.5574732481877804, + "metric": 0.7954780807732137, + "rescued": 67, + "harmed": 682, + "missed_rescue": 77, + "avoided_harm": 1510, + "selected_both_correct": 2274, + "selected_both_wrong": 207 + }, + { + "budget": 0.55, + "threshold": -10.71943473815918, + "operator": "ge", + "actual_fraction": 0.5629962029685882, + "metric": 0.7944425267518123, + "rescued": 69, + "harmed": 690, + "missed_rescue": 75, + "avoided_harm": 1502, + "selected_both_correct": 2296, + "selected_both_wrong": 207 + }, + { + "budget": 0.56, + "threshold": -10.831548690795898, + "operator": "ge", + "actual_fraction": 0.5693821194338972, + "metric": 0.7925440110459095, + "rescued": 70, + "harmed": 702, + "missed_rescue": 74, + "avoided_harm": 1490, + "selected_both_correct": 2319, + "selected_both_wrong": 208 + }, + { + "budget": 0.57, + "threshold": -11.046300888061523, + "operator": "ge", + "actual_fraction": 0.5800828443217121, + "metric": 0.7882292026234036, + "rescued": 72, + "harmed": 729, + "missed_rescue": 72, + "avoided_harm": 1463, + "selected_both_correct": 2348, + "selected_both_wrong": 212 + }, + { + "budget": 0.58, + "threshold": -11.17777156829834, + "operator": "ge", + "actual_fraction": 0.5894028305143252, + "metric": 0.7851225405591992, + "rescued": 74, + "harmed": 749, + "missed_rescue": 70, + "avoided_harm": 1443, + "selected_both_correct": 2374, + "selected_both_wrong": 218 + }, + { + "budget": 0.59, + "threshold": -11.355005264282227, + "operator": "ge", + "actual_fraction": 0.5992405937176389, + "metric": 0.7813255091473938, + "rescued": 75, + "harmed": 772, + "missed_rescue": 69, + "avoided_harm": 1420, + "selected_both_correct": 2402, + "selected_both_wrong": 223 + }, + { + "budget": 0.6, + "threshold": -11.538931846618652, + "operator": "ge", + "actual_fraction": 0.6104590956161546, + "metric": 0.7770107007248878, + "rescued": 76, + "harmed": 798, + "missed_rescue": 68, + "avoided_harm": 1394, + "selected_both_correct": 2432, + "selected_both_wrong": 231 + }, + { + "budget": 0.61, + "threshold": -11.703760147094727, + "operator": "ge", + "actual_fraction": 0.6197790818087677, + "metric": 0.7726958923023818, + "rescued": 78, + "harmed": 825, + "missed_rescue": 66, + "avoided_harm": 1367, + "selected_both_correct": 2454, + "selected_both_wrong": 234 + }, + { + "budget": 0.62, + "threshold": -11.831314086914062, + "operator": "ge", + "actual_fraction": 0.6280635139799793, + "metric": 0.7688988608905765, + "rescued": 78, + "harmed": 847, + "missed_rescue": 66, + "avoided_harm": 1345, + "selected_both_correct": 2479, + "selected_both_wrong": 235 + }, + { + "budget": 0.63, + "threshold": -12.021162986755371, + "operator": "ge", + "actual_fraction": 0.6366931308249913, + "metric": 0.7651018294787711, + "rescued": 81, + "harmed": 872, + "missed_rescue": 63, + "avoided_harm": 1320, + "selected_both_correct": 2496, + "selected_both_wrong": 240 + }, + { + "budget": 0.64, + "threshold": -12.189552307128906, + "operator": "ge", + "actual_fraction": 0.6465308940283051, + "metric": 0.7602692440455644, + "rescued": 82, + "harmed": 901, + "missed_rescue": 62, + "avoided_harm": 1291, + "selected_both_correct": 2521, + "selected_both_wrong": 242 + }, + { + "budget": 0.65, + "threshold": -12.395832061767578, + "operator": "ge", + "actual_fraction": 0.6582671729375216, + "metric": 0.7561270279599586, + "rescued": 85, + "harmed": 928, + "missed_rescue": 59, + "avoided_harm": 1264, + "selected_both_correct": 2556, + "selected_both_wrong": 245 + }, + { + "budget": 0.66, + "threshold": -12.56098461151123, + "operator": "ge", + "actual_fraction": 0.6660338280980325, + "metric": 0.7526751812219538, + "rescued": 85, + "harmed": 948, + "missed_rescue": 59, + "avoided_harm": 1244, + "selected_both_correct": 2578, + "selected_both_wrong": 248 + }, + { + "budget": 0.67, + "threshold": -12.731056213378906, + "operator": "ge", + "actual_fraction": 0.675698998964446, + "metric": 0.7480151881256473, + "rescued": 86, + "harmed": 976, + "missed_rescue": 58, + "avoided_harm": 1216, + "selected_both_correct": 2604, + "selected_both_wrong": 249 + }, + { + "budget": 0.68, + "threshold": -12.946990966796875, + "operator": "ge", + "actual_fraction": 0.6862271315153607, + "metric": 0.7435277873662409, + "rescued": 86, + "harmed": 1002, + "missed_rescue": 58, + "avoided_harm": 1190, + "selected_both_correct": 2638, + "selected_both_wrong": 250 + }, + { + "budget": 0.69, + "threshold": -13.060778617858887, + "operator": "ge", + "actual_fraction": 0.691404901622368, + "metric": 0.7416292716603383, + "rescued": 87, + "harmed": 1014, + "missed_rescue": 57, + "avoided_harm": 1178, + "selected_both_correct": 2650, + "selected_both_wrong": 255 + }, + { + "budget": 0.7, + "threshold": -13.237710952758789, + "operator": "ge", + "actual_fraction": 0.699344149119779, + "metric": 0.7381774249223334, + "rescued": 88, + "harmed": 1035, + "missed_rescue": 56, + "avoided_harm": 1157, + "selected_both_correct": 2671, + "selected_both_wrong": 258 + }, + { + "budget": 0.71, + "threshold": -13.452357292175293, + "operator": "ge", + "actual_fraction": 0.7076285812909907, + "metric": 0.7345529858474283, + "rescued": 89, + "harmed": 1057, + "missed_rescue": 55, + "avoided_harm": 1135, + "selected_both_correct": 2689, + "selected_both_wrong": 265 + }, + { + "budget": 0.72, + "threshold": -13.714129447937012, + "operator": "ge", + "actual_fraction": 0.7197100448740076, + "metric": 0.7298929927511218, + "rescued": 92, + "harmed": 1087, + "missed_rescue": 52, + "avoided_harm": 1105, + "selected_both_correct": 2720, + "selected_both_wrong": 271 + }, + { + "budget": 0.73, + "threshold": -13.887096405029297, + "operator": "ge", + "actual_fraction": 0.7276492923714187, + "metric": 0.7255781843286158, + "rescued": 92, + "harmed": 1112, + "missed_rescue": 52, + "avoided_harm": 1080, + "selected_both_correct": 2736, + "selected_both_wrong": 276 + }, + { + "budget": 0.74, + "threshold": -14.114652633666992, + "operator": "ge", + "actual_fraction": 0.7380048325854333, + "metric": 0.7198826372109078, + "rescued": 92, + "harmed": 1145, + "missed_rescue": 52, + "avoided_harm": 1047, + "selected_both_correct": 2761, + "selected_both_wrong": 278 + }, + { + "budget": 0.75, + "threshold": -14.356359481811523, + "operator": "ge", + "actual_fraction": 0.7469796341042457, + "metric": 0.7131515360717984, + "rescued": 92, + "harmed": 1184, + "missed_rescue": 52, + "avoided_harm": 1008, + "selected_both_correct": 2770, + "selected_both_wrong": 282 + }, + { + "budget": 0.76, + "threshold": -14.591072082519531, + "operator": "ge", + "actual_fraction": 0.7559544356230583, + "metric": 0.7091819123230928, + "rescued": 96, + "harmed": 1211, + "missed_rescue": 48, + "avoided_harm": 981, + "selected_both_correct": 2789, + "selected_both_wrong": 284 + }, + { + "budget": 0.77, + "threshold": -15.056472778320312, + "operator": "ge", + "actual_fraction": 0.7758025543665861, + "metric": 0.6979634104245771, + "rescued": 98, + "harmed": 1278, + "missed_rescue": 46, + "avoided_harm": 914, + "selected_both_correct": 2829, + "selected_both_wrong": 290 + }, + { + "budget": 0.78, + "threshold": -15.384273529052734, + "operator": "ge", + "actual_fraction": 0.7884017949603037, + "metric": 0.6915774939592683, + "rescued": 101, + "harmed": 1318, + "missed_rescue": 43, + "avoided_harm": 874, + "selected_both_correct": 2855, + "selected_both_wrong": 294 + }, + { + "budget": 0.79, + "threshold": -15.68124008178711, + "operator": "ge", + "actual_fraction": 0.7991025198481188, + "metric": 0.6845012081463583, + "rescued": 103, + "harmed": 1361, + "missed_rescue": 41, + "avoided_harm": 831, + "selected_both_correct": 2869, + "selected_both_wrong": 297 + }, + { + "budget": 0.8, + "threshold": -15.871927261352539, + "operator": "ge", + "actual_fraction": 0.8063513979979289, + "metric": 0.6796686227131515, + "rescued": 104, + "harmed": 1390, + "missed_rescue": 40, + "avoided_harm": 802, + "selected_both_correct": 2880, + "selected_both_wrong": 298 + }, + { + "budget": 0.81, + "threshold": -16.09673500061035, + "operator": "ge", + "actual_fraction": 0.81429064549534, + "metric": 0.6741456679323438, + "rescued": 104, + "harmed": 1422, + "missed_rescue": 40, + "avoided_harm": 770, + "selected_both_correct": 2890, + "selected_both_wrong": 302 + }, + { + "budget": 0.82, + "threshold": -16.361101150512695, + "operator": "ge", + "actual_fraction": 0.8203313772868485, + "metric": 0.6703486365205384, + "rescued": 105, + "harmed": 1445, + "missed_rescue": 39, + "avoided_harm": 747, + "selected_both_correct": 2899, + "selected_both_wrong": 304 + }, + { + "budget": 0.83, + "threshold": -16.775108337402344, + "operator": "ge", + "actual_fraction": 0.8339661719019675, + "metric": 0.6636175353814291, + "rescued": 111, + "harmed": 1490, + "missed_rescue": 33, + "avoided_harm": 702, + "selected_both_correct": 2921, + "selected_both_wrong": 310 + }, + { + "budget": 0.84, + "threshold": -17.00735855102539, + "operator": "ge", + "actual_fraction": 0.8412150500517777, + "metric": 0.6594753192958233, + "rescued": 112, + "harmed": 1515, + "missed_rescue": 32, + "avoided_harm": 677, + "selected_both_correct": 2931, + "selected_both_wrong": 316 + }, + { + "budget": 0.85, + "threshold": -17.19187355041504, + "operator": "ge", + "actual_fraction": 0.8465654124956852, + "metric": 0.656368657231619, + "rescued": 112, + "harmed": 1533, + "missed_rescue": 32, + "avoided_harm": 659, + "selected_both_correct": 2938, + "selected_both_wrong": 322 + }, + { + "budget": 0.86, + "threshold": -17.789541244506836, + "operator": "ge", + "actual_fraction": 0.8612357611322057, + "metric": 0.6456679323438039, + "rescued": 112, + "harmed": 1595, + "missed_rescue": 32, + "avoided_harm": 597, + "selected_both_correct": 2956, + "selected_both_wrong": 327 + }, + { + "budget": 0.87, + "threshold": -18.021289825439453, + "operator": "ge", + "actual_fraction": 0.8676216775975146, + "metric": 0.6416983085950984, + "rescued": 114, + "harmed": 1620, + "missed_rescue": 30, + "avoided_harm": 572, + "selected_both_correct": 2962, + "selected_both_wrong": 331 + }, + { + "budget": 0.88, + "threshold": -18.256092071533203, + "operator": "ge", + "actual_fraction": 0.8745253710735243, + "metric": 0.637901277183293, + "rescued": 117, + "harmed": 1645, + "missed_rescue": 27, + "avoided_harm": 547, + "selected_both_correct": 2969, + "selected_both_wrong": 336 + }, + { + "budget": 0.89, + "threshold": -18.753929138183594, + "operator": "ge", + "actual_fraction": 0.8866068346565412, + "metric": 0.6289264756644805, + "rescued": 119, + "harmed": 1699, + "missed_rescue": 25, + "avoided_harm": 493, + "selected_both_correct": 2978, + "selected_both_wrong": 341 + }, + { + "budget": 0.9, + "threshold": -19.046903610229492, + "operator": "ge", + "actual_fraction": 0.8919571971004487, + "metric": 0.624956851915775, + "rescued": 120, + "harmed": 1723, + "missed_rescue": 24, + "avoided_harm": 469, + "selected_both_correct": 2980, + "selected_both_wrong": 345 + }, + { + "budget": 0.91, + "threshold": -19.4178466796875, + "operator": "ge", + "actual_fraction": 0.9000690369347601, + "metric": 0.619951674145668, + "rescued": 122, + "harmed": 1754, + "missed_rescue": 22, + "avoided_harm": 438, + "selected_both_correct": 2989, + "selected_both_wrong": 350 + }, + { + "budget": 0.92, + "threshold": -19.944700241088867, + "operator": "ge", + "actual_fraction": 0.9095616154642734, + "metric": 0.613565757680359, + "rescued": 126, + "harmed": 1795, + "missed_rescue": 18, + "avoided_harm": 397, + "selected_both_correct": 2995, + "selected_both_wrong": 354 + }, + { + "budget": 0.93, + "threshold": -20.247802734375, + "operator": "ge", + "actual_fraction": 0.9140490162236796, + "metric": 0.6097687262685537, + "rescued": 126, + "harmed": 1817, + "missed_rescue": 18, + "avoided_harm": 375, + "selected_both_correct": 2996, + "selected_both_wrong": 357 + }, + { + "budget": 0.94, + "threshold": -21.42934226989746, + "operator": "ge", + "actual_fraction": 0.9316534345875043, + "metric": 0.5968243010010356, + "rescued": 129, + "harmed": 1895, + "missed_rescue": 15, + "avoided_harm": 297, + "selected_both_correct": 3006, + "selected_both_wrong": 368 + }, + { + "budget": 0.95, + "threshold": -22.04431915283203, + "operator": "ge", + "actual_fraction": 0.9433897134967207, + "metric": 0.587849499482223, + "rescued": 131, + "harmed": 1949, + "missed_rescue": 13, + "avoided_harm": 243, + "selected_both_correct": 3012, + "selected_both_wrong": 374 + }, + { + "budget": 0.96, + "threshold": -23.509872436523438, + "operator": "ge", + "actual_fraction": 0.9609941318605454, + "metric": 0.5752502588885053, + "rescued": 134, + "harmed": 2025, + "missed_rescue": 10, + "avoided_harm": 167, + "selected_both_correct": 3021, + "selected_both_wrong": 388 + }, + { + "budget": 0.97, + "threshold": -24.155170440673828, + "operator": "ge", + "actual_fraction": 0.9675526406627546, + "metric": 0.5705902657921988, + "rescued": 136, + "harmed": 2054, + "missed_rescue": 8, + "avoided_harm": 138, + "selected_both_correct": 3023, + "selected_both_wrong": 393 + }, + { + "budget": 0.98, + "threshold": -25.53754234313965, + "operator": "ge", + "actual_fraction": 0.9770452191922678, + "metric": 0.5629962029685882, + "rescued": 136, + "harmed": 2098, + "missed_rescue": 8, + "avoided_harm": 94, + "selected_both_correct": 3025, + "selected_both_wrong": 402 + }, + { + "budget": 0.99, + "threshold": -27.78575325012207, + "operator": "ge", + "actual_fraction": 0.9892992751121851, + "metric": 0.5541939937866759, + "rescued": 139, + "harmed": 2152, + "missed_rescue": 5, + "avoided_harm": 40, + "selected_both_correct": 3030, + "selected_both_wrong": 411 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5481532619951675, + "rescued": 144, + "harmed": 2192, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 3032, + "selected_both_wrong": 426 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 659, + "rescuable": 2, + "harmful": 528, + "both_wrong": 11, + "unit": "sample", + "total": 1200 + }, + "test": { + "both_correct": 3032, + "rescuable": 144, + "harmful": 2192, + "both_wrong": 426, + "unit": "sample", + "total": 5794 + } + }, + "small": { + "validation": 0.9891666666666666, + "test": 0.9016223679668622 + }, + "large": { + "validation": 0.5508333333333333, + "test": 0.5481532619951675 + }, + "learned_minus_confidence_test": 0.012771832930617899, + "learned_minus_small_test": -0.0010355540214014303, + "learned_minus_large_test": 0.35243355195029336 + }, + { + "key": [ + "grefcoco", + "groundingdino", + "MiniCPM" + ], + "expert_display": "GroundingDINO", + "run": "shared_12_groundingdino_MiniCPM", + "validation_rows": 14229, + "test_rows": 35263, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detection_count", + "max_score", + "mean_score", + "query_count", + "detection_count_valid", + "max_score_valid", + "mean_score_valid", + "query_count_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/groundingdino/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 99, + "validation_area_0_25": 0.5566729917773561, + "validation_point": { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.6372197624569541, + "rescued": 2529, + "harmed": 1438, + "missed_rescue": 60, + "avoided_harm": 3, + "selected_both_correct": 6534, + "selected_both_wrong": 3585 + }, + "test_point": { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9873805405098829, + "metric": 0.4968380455434875, + "rescued": 10590, + "harmed": 2481, + "missed_rescue": 133, + "avoided_harm": 19, + "selected_both_correct": 6845, + "selected_both_wrong": 14902 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.5605453650994447, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2589, + "avoided_harm": 1441, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.1584138125181198, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.5602642490688031, + "rescued": 0, + "harmed": 4, + "missed_rescue": 2589, + "avoided_harm": 1437, + "selected_both_correct": 138, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -0.21403978765010834, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.5599128540305011, + "rescued": 0, + "harmed": 9, + "missed_rescue": 2589, + "avoided_harm": 1432, + "selected_both_correct": 275, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -0.24253316223621368, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.559561458992199, + "rescued": 0, + "harmed": 14, + "missed_rescue": 2589, + "avoided_harm": 1427, + "selected_both_correct": 409, + "selected_both_wrong": 3 + }, + { + "budget": 0.04, + "threshold": -0.2599165141582489, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.5589289479232553, + "rescued": 0, + "harmed": 23, + "missed_rescue": 2589, + "avoided_harm": 1418, + "selected_both_correct": 542, + "selected_both_wrong": 4 + }, + { + "budget": 0.05, + "threshold": -0.2725968658924103, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.5583667158619721, + "rescued": 1, + "harmed": 32, + "missed_rescue": 2588, + "avoided_harm": 1409, + "selected_both_correct": 666, + "selected_both_wrong": 12 + }, + { + "budget": 0.06, + "threshold": -0.2842792272567749, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.5578747628083491, + "rescued": 2, + "harmed": 40, + "missed_rescue": 2587, + "avoided_harm": 1401, + "selected_both_correct": 794, + "selected_both_wrong": 17 + }, + { + "budget": 0.07, + "threshold": -0.2935555577278137, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.5569611357087638, + "rescued": 3, + "harmed": 54, + "missed_rescue": 2586, + "avoided_harm": 1387, + "selected_both_correct": 911, + "selected_both_wrong": 28 + }, + { + "budget": 0.08, + "threshold": -0.30114513635635376, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.5566800196781222, + "rescued": 8, + "harmed": 63, + "missed_rescue": 2581, + "avoided_harm": 1378, + "selected_both_correct": 1032, + "selected_both_wrong": 35 + }, + { + "budget": 0.09, + "threshold": -0.308601438999176, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.5562583456321597, + "rescued": 14, + "harmed": 75, + "missed_rescue": 2575, + "avoided_harm": 1366, + "selected_both_correct": 1140, + "selected_both_wrong": 51 + }, + { + "budget": 0.1, + "threshold": -0.3146189749240875, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.555625834563216, + "rescued": 18, + "harmed": 88, + "missed_rescue": 2571, + "avoided_harm": 1353, + "selected_both_correct": 1249, + "selected_both_wrong": 67 + }, + { + "budget": 0.11, + "threshold": -0.32043084502220154, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.555625834563216, + "rescued": 24, + "harmed": 94, + "missed_rescue": 2565, + "avoided_harm": 1347, + "selected_both_correct": 1360, + "selected_both_wrong": 87 + }, + { + "budget": 0.12, + "threshold": -0.32644838094711304, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.5553447185325743, + "rescued": 35, + "harmed": 109, + "missed_rescue": 2554, + "avoided_harm": 1332, + "selected_both_correct": 1461, + "selected_both_wrong": 102 + }, + { + "budget": 0.13, + "threshold": -0.33121857047080994, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.5553447185325743, + "rescued": 46, + "harmed": 120, + "missed_rescue": 2543, + "avoided_harm": 1321, + "selected_both_correct": 1562, + "selected_both_wrong": 121 + }, + { + "budget": 0.14, + "threshold": -0.33570271730422974, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.5554149975402347, + "rescued": 57, + "harmed": 130, + "missed_rescue": 2532, + "avoided_harm": 1311, + "selected_both_correct": 1662, + "selected_both_wrong": 143 + }, + { + "budget": 0.15, + "threshold": -0.3407592475414276, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.5549933234942722, + "rescued": 67, + "harmed": 146, + "missed_rescue": 2522, + "avoided_harm": 1295, + "selected_both_correct": 1754, + "selected_both_wrong": 167 + }, + { + "budget": 0.16, + "threshold": -0.34456855058670044, + "operator": "ge", + "actual_fraction": 0.15995502143509735, + "metric": 0.5546419284559702, + "rescued": 81, + "harmed": 165, + "missed_rescue": 2508, + "avoided_harm": 1276, + "selected_both_correct": 1844, + "selected_both_wrong": 186 + }, + { + "budget": 0.17, + "threshold": -0.3487326204776764, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.5551338815095931, + "rescued": 99, + "harmed": 176, + "missed_rescue": 2490, + "avoided_harm": 1265, + "selected_both_correct": 1931, + "selected_both_wrong": 212 + }, + { + "budget": 0.18, + "threshold": -0.35280776023864746, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.5553447185325743, + "rescued": 114, + "harmed": 188, + "missed_rescue": 2475, + "avoided_harm": 1253, + "selected_both_correct": 2017, + "selected_both_wrong": 242 + }, + { + "budget": 0.19, + "threshold": -0.3564819097518921, + "operator": "ge", + "actual_fraction": 0.18996415770609318, + "metric": 0.555274439524914, + "rescued": 126, + "harmed": 201, + "missed_rescue": 2463, + "avoided_harm": 1240, + "selected_both_correct": 2103, + "selected_both_wrong": 273 + }, + { + "budget": 0.2, + "threshold": -0.35992002487182617, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.5554852765478951, + "rescued": 142, + "harmed": 214, + "missed_rescue": 2447, + "avoided_harm": 1227, + "selected_both_correct": 2186, + "selected_both_wrong": 303 + }, + { + "budget": 0.21, + "threshold": -0.36340460181236267, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.5566097406704618, + "rescued": 166, + "harmed": 222, + "missed_rescue": 2423, + "avoided_harm": 1219, + "selected_both_correct": 2269, + "selected_both_wrong": 331 + }, + { + "budget": 0.22, + "threshold": -0.3674520254135132, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.5566800196781222, + "rescued": 182, + "harmed": 237, + "missed_rescue": 2407, + "avoided_harm": 1204, + "selected_both_correct": 2349, + "selected_both_wrong": 362 + }, + { + "budget": 0.23, + "threshold": -0.37108901143074036, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.556820577693443, + "rescued": 197, + "harmed": 250, + "missed_rescue": 2392, + "avoided_harm": 1191, + "selected_both_correct": 2436, + "selected_both_wrong": 389 + }, + { + "budget": 0.24, + "threshold": -0.3745429813861847, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.5575233677700471, + "rescued": 219, + "harmed": 262, + "missed_rescue": 2370, + "avoided_harm": 1179, + "selected_both_correct": 2514, + "selected_both_wrong": 419 + }, + { + "budget": 0.25, + "threshold": -0.3773327171802521, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.5572422517394054, + "rescued": 233, + "harmed": 280, + "missed_rescue": 2356, + "avoided_harm": 1161, + "selected_both_correct": 2590, + "selected_both_wrong": 454 + }, + { + "budget": 0.26, + "threshold": -0.3803459703922272, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.5584369948696324, + "rescued": 260, + "harmed": 290, + "missed_rescue": 2329, + "avoided_harm": 1151, + "selected_both_correct": 2663, + "selected_both_wrong": 486 + }, + { + "budget": 0.27, + "threshold": -0.38361650705337524, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.5585775528849533, + "rescued": 278, + "harmed": 306, + "missed_rescue": 2311, + "avoided_harm": 1135, + "selected_both_correct": 2737, + "selected_both_wrong": 520 + }, + { + "budget": 0.28, + "threshold": -0.38652583956718445, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.5589992269309157, + "rescued": 298, + "harmed": 320, + "missed_rescue": 2291, + "avoided_harm": 1121, + "selected_both_correct": 2802, + "selected_both_wrong": 564 + }, + { + "budget": 0.29, + "threshold": -0.38952407240867615, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.5599831330381615, + "rescued": 323, + "harmed": 331, + "missed_rescue": 2266, + "avoided_harm": 1110, + "selected_both_correct": 2873, + "selected_both_wrong": 599 + }, + { + "budget": 0.3, + "threshold": -0.39262744784355164, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.5608264811300865, + "rescued": 348, + "harmed": 344, + "missed_rescue": 2241, + "avoided_harm": 1097, + "selected_both_correct": 2941, + "selected_both_wrong": 635 + }, + { + "budget": 0.31, + "threshold": -0.3954668939113617, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.5613184341837093, + "rescued": 370, + "harmed": 359, + "missed_rescue": 2219, + "avoided_harm": 1082, + "selected_both_correct": 3011, + "selected_both_wrong": 670 + }, + { + "budget": 0.32, + "threshold": -0.3986983895301819, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.5613887131913697, + "rescued": 394, + "harmed": 382, + "missed_rescue": 2195, + "avoided_harm": 1059, + "selected_both_correct": 3076, + "selected_both_wrong": 701 + }, + { + "budget": 0.33, + "threshold": -0.4014112949371338, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.5616698292220114, + "rescued": 415, + "harmed": 399, + "missed_rescue": 2174, + "avoided_harm": 1042, + "selected_both_correct": 3145, + "selected_both_wrong": 736 + }, + { + "budget": 0.34, + "threshold": -0.4042409658432007, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.562302340290955, + "rescued": 437, + "harmed": 412, + "missed_rescue": 2152, + "avoided_harm": 1029, + "selected_both_correct": 3220, + "selected_both_wrong": 768 + }, + { + "budget": 0.35, + "threshold": -0.40724706649780273, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.5628645723522384, + "rescued": 461, + "harmed": 428, + "missed_rescue": 2128, + "avoided_harm": 1013, + "selected_both_correct": 3288, + "selected_both_wrong": 803 + }, + { + "budget": 0.36, + "threshold": -0.40968337655067444, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.5624428983062759, + "rescued": 478, + "harmed": 451, + "missed_rescue": 2111, + "avoided_harm": 990, + "selected_both_correct": 3361, + "selected_both_wrong": 832 + }, + { + "budget": 0.37, + "threshold": -0.4123799502849579, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.5635673624288425, + "rescued": 508, + "harmed": 465, + "missed_rescue": 2081, + "avoided_harm": 976, + "selected_both_correct": 3425, + "selected_both_wrong": 866 + }, + { + "budget": 0.38, + "threshold": -0.41514647006988525, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.5650432215897112, + "rescued": 538, + "harmed": 474, + "missed_rescue": 2051, + "avoided_harm": 967, + "selected_both_correct": 3487, + "selected_both_wrong": 908 + }, + { + "budget": 0.39, + "threshold": -0.4179389476776123, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.565183779605032, + "rescued": 557, + "harmed": 491, + "missed_rescue": 2032, + "avoided_harm": 950, + "selected_both_correct": 3556, + "selected_both_wrong": 945 + }, + { + "budget": 0.4, + "threshold": -0.42099592089653015, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.5658865696816361, + "rescued": 580, + "harmed": 504, + "missed_rescue": 2009, + "avoided_harm": 937, + "selected_both_correct": 3625, + "selected_both_wrong": 982 + }, + { + "budget": 0.41, + "threshold": -0.4243226647377014, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.5672218708271839, + "rescued": 614, + "harmed": 519, + "missed_rescue": 1975, + "avoided_harm": 922, + "selected_both_correct": 3695, + "selected_both_wrong": 1005 + }, + { + "budget": 0.42, + "threshold": -0.4275209903717041, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.5686977299880526, + "rescued": 645, + "harmed": 529, + "missed_rescue": 1944, + "avoided_harm": 912, + "selected_both_correct": 3759, + "selected_both_wrong": 1043 + }, + { + "budget": 0.43, + "threshold": -0.4307159185409546, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.5695410780799776, + "rescued": 672, + "harmed": 544, + "missed_rescue": 1917, + "avoided_harm": 897, + "selected_both_correct": 3828, + "selected_both_wrong": 1074 + }, + { + "budget": 0.44, + "threshold": -0.4336964190006256, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.5705952631948837, + "rescued": 702, + "harmed": 559, + "missed_rescue": 1887, + "avoided_harm": 882, + "selected_both_correct": 3892, + "selected_both_wrong": 1107 + }, + { + "budget": 0.45, + "threshold": -0.4373021125793457, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.5709466582331858, + "rescued": 725, + "harmed": 577, + "missed_rescue": 1864, + "avoided_harm": 864, + "selected_both_correct": 3955, + "selected_both_wrong": 1146 + }, + { + "budget": 0.46, + "threshold": -0.4404309391975403, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.5722116803710732, + "rescued": 759, + "harmed": 593, + "missed_rescue": 1830, + "avoided_harm": 848, + "selected_both_correct": 4019, + "selected_both_wrong": 1174 + }, + { + "budget": 0.47, + "threshold": -0.44344601035118103, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.5731955864783189, + "rescued": 784, + "harmed": 604, + "missed_rescue": 1805, + "avoided_harm": 837, + "selected_both_correct": 4091, + "selected_both_wrong": 1208 + }, + { + "budget": 0.48, + "threshold": -0.4465356171131134, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.5732658654859794, + "rescued": 805, + "harmed": 624, + "missed_rescue": 1784, + "avoided_harm": 817, + "selected_both_correct": 4159, + "selected_both_wrong": 1241 + }, + { + "budget": 0.49, + "threshold": -0.44965147972106934, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.5741794925855647, + "rescued": 836, + "harmed": 642, + "missed_rescue": 1753, + "avoided_harm": 799, + "selected_both_correct": 4222, + "selected_both_wrong": 1272 + }, + { + "budget": 0.5, + "threshold": -0.453143447637558, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.574741724646848, + "rescued": 860, + "harmed": 658, + "missed_rescue": 1729, + "avoided_harm": 783, + "selected_both_correct": 4278, + "selected_both_wrong": 1318 + }, + { + "budget": 0.51, + "threshold": -0.4565357565879822, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.5757256307540938, + "rescued": 892, + "harmed": 676, + "missed_rescue": 1697, + "avoided_harm": 765, + "selected_both_correct": 4336, + "selected_both_wrong": 1352 + }, + { + "budget": 0.52, + "threshold": -0.45961302518844604, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.5763581418230375, + "rescued": 916, + "harmed": 691, + "missed_rescue": 1673, + "avoided_harm": 750, + "selected_both_correct": 4403, + "selected_both_wrong": 1389 + }, + { + "budget": 0.53, + "threshold": -0.46368208527565, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.5774826059456041, + "rescued": 948, + "harmed": 707, + "missed_rescue": 1641, + "avoided_harm": 734, + "selected_both_correct": 4468, + "selected_both_wrong": 1418 + }, + { + "budget": 0.54, + "threshold": -0.46793216466903687, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.5792395811371144, + "rescued": 986, + "harmed": 720, + "missed_rescue": 1603, + "avoided_harm": 721, + "selected_both_correct": 4528, + "selected_both_wrong": 1449 + }, + { + "budget": 0.55, + "threshold": -0.4718930721282959, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.5795909761754164, + "rescued": 1010, + "harmed": 739, + "missed_rescue": 1579, + "avoided_harm": 702, + "selected_both_correct": 4579, + "selected_both_wrong": 1497 + }, + { + "budget": 0.56, + "threshold": -0.47629430890083313, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.5794504181600956, + "rescued": 1031, + "harmed": 762, + "missed_rescue": 1558, + "avoided_harm": 679, + "selected_both_correct": 4644, + "selected_both_wrong": 1531 + }, + { + "budget": 0.57, + "threshold": -0.4799485504627228, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.5802234872443601, + "rescued": 1060, + "harmed": 780, + "missed_rescue": 1529, + "avoided_harm": 661, + "selected_both_correct": 4700, + "selected_both_wrong": 1570 + }, + { + "budget": 0.58, + "threshold": -0.4838995635509491, + "operator": "ge", + "actual_fraction": 0.5799423712137185, + "metric": 0.5812776723592663, + "rescued": 1093, + "harmed": 798, + "missed_rescue": 1496, + "avoided_harm": 643, + "selected_both_correct": 4756, + "selected_both_wrong": 1605 + }, + { + "budget": 0.59, + "threshold": -0.4886912405490875, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.5816290673975684, + "rescued": 1120, + "harmed": 820, + "missed_rescue": 1469, + "avoided_harm": 621, + "selected_both_correct": 4815, + "selected_both_wrong": 1640 + }, + { + "budget": 0.6, + "threshold": -0.49332454800605774, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.5819804624358704, + "rescued": 1142, + "harmed": 837, + "missed_rescue": 1447, + "avoided_harm": 604, + "selected_both_correct": 4882, + "selected_both_wrong": 1676 + }, + { + "budget": 0.61, + "threshold": -0.49750789999961853, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.582261578466512, + "rescued": 1166, + "harmed": 857, + "missed_rescue": 1423, + "avoided_harm": 584, + "selected_both_correct": 4941, + "selected_both_wrong": 1715 + }, + { + "budget": 0.62, + "threshold": -0.5024909377098083, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.5823318574741725, + "rescued": 1183, + "harmed": 873, + "missed_rescue": 1406, + "avoided_harm": 568, + "selected_both_correct": 5010, + "selected_both_wrong": 1755 + }, + { + "budget": 0.63, + "threshold": -0.5067122578620911, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.5824021364818329, + "rescued": 1205, + "harmed": 894, + "missed_rescue": 1384, + "avoided_harm": 547, + "selected_both_correct": 5076, + "selected_both_wrong": 1789 + }, + { + "budget": 0.64, + "threshold": -0.512323796749115, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.5825426944971537, + "rescued": 1223, + "harmed": 910, + "missed_rescue": 1366, + "avoided_harm": 531, + "selected_both_correct": 5140, + "selected_both_wrong": 1833 + }, + { + "budget": 0.65, + "threshold": -0.5170255303382874, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.5833157635814182, + "rescued": 1250, + "harmed": 926, + "missed_rescue": 1339, + "avoided_harm": 515, + "selected_both_correct": 5200, + "selected_both_wrong": 1872 + }, + { + "budget": 0.66, + "threshold": -0.5215396881103516, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.5833157635814182, + "rescued": 1269, + "harmed": 945, + "missed_rescue": 1320, + "avoided_harm": 496, + "selected_both_correct": 5264, + "selected_both_wrong": 1913 + }, + { + "budget": 0.67, + "threshold": -0.5264327526092529, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.5826129735048141, + "rescued": 1288, + "harmed": 974, + "missed_rescue": 1301, + "avoided_harm": 467, + "selected_both_correct": 5321, + "selected_both_wrong": 1950 + }, + { + "budget": 0.68, + "threshold": -0.5312389135360718, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.583104926558437, + "rescued": 1316, + "harmed": 995, + "missed_rescue": 1273, + "avoided_harm": 446, + "selected_both_correct": 5379, + "selected_both_wrong": 1985 + }, + { + "budget": 0.69, + "threshold": -0.5368650555610657, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.5835266006043994, + "rescued": 1336, + "harmed": 1009, + "missed_rescue": 1253, + "avoided_harm": 432, + "selected_both_correct": 5445, + "selected_both_wrong": 2028 + }, + { + "budget": 0.7, + "threshold": -0.5431069135665894, + "operator": "ge", + "actual_fraction": 0.6999789162977019, + "metric": 0.5843699486963244, + "rescued": 1363, + "harmed": 1024, + "missed_rescue": 1226, + "avoided_harm": 417, + "selected_both_correct": 5492, + "selected_both_wrong": 2081 + }, + { + "budget": 0.71, + "threshold": -0.5487136244773865, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.5852835757959097, + "rescued": 1393, + "harmed": 1041, + "missed_rescue": 1196, + "avoided_harm": 400, + "selected_both_correct": 5550, + "selected_both_wrong": 2118 + }, + { + "budget": 0.72, + "threshold": -0.5549414753913879, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.5849321807576077, + "rescued": 1415, + "harmed": 1068, + "missed_rescue": 1174, + "avoided_harm": 373, + "selected_both_correct": 5611, + "selected_both_wrong": 2150 + }, + { + "budget": 0.73, + "threshold": -0.5604677200317383, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.5845105067116453, + "rescued": 1432, + "harmed": 1091, + "missed_rescue": 1157, + "avoided_harm": 350, + "selected_both_correct": 5683, + "selected_both_wrong": 2181 + }, + { + "budget": 0.74, + "threshold": -0.5666090250015259, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.584651064726966, + "rescued": 1452, + "harmed": 1109, + "missed_rescue": 1137, + "avoided_harm": 332, + "selected_both_correct": 5738, + "selected_both_wrong": 2230 + }, + { + "budget": 0.75, + "threshold": -0.574927568435669, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.5847213437346265, + "rescued": 1473, + "harmed": 1129, + "missed_rescue": 1116, + "avoided_harm": 312, + "selected_both_correct": 5796, + "selected_both_wrong": 2273 + }, + { + "budget": 0.76, + "threshold": -0.5812664031982422, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.5850727387729285, + "rescued": 1499, + "harmed": 1150, + "missed_rescue": 1090, + "avoided_harm": 291, + "selected_both_correct": 5856, + "selected_both_wrong": 2309 + }, + { + "budget": 0.77, + "threshold": -0.5883752107620239, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.5853538548035702, + "rescued": 1522, + "harmed": 1169, + "missed_rescue": 1067, + "avoided_harm": 272, + "selected_both_correct": 5923, + "selected_both_wrong": 2342 + }, + { + "budget": 0.78, + "threshold": -0.5972744226455688, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.5852132967882494, + "rescued": 1540, + "harmed": 1189, + "missed_rescue": 1049, + "avoided_harm": 252, + "selected_both_correct": 5976, + "selected_both_wrong": 2393 + }, + { + "budget": 0.79, + "threshold": -0.6058458089828491, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.5857755288495327, + "rescued": 1569, + "harmed": 1210, + "missed_rescue": 1020, + "avoided_harm": 231, + "selected_both_correct": 6037, + "selected_both_wrong": 2424 + }, + { + "budget": 0.8, + "threshold": -0.6140543818473816, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.5853538548035702, + "rescued": 1596, + "harmed": 1243, + "missed_rescue": 993, + "avoided_harm": 198, + "selected_both_correct": 6083, + "selected_both_wrong": 2461 + }, + { + "budget": 0.81, + "threshold": -0.6230268478393555, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.5852132967882494, + "rescued": 1619, + "harmed": 1268, + "missed_rescue": 970, + "avoided_harm": 173, + "selected_both_correct": 6141, + "selected_both_wrong": 2497 + }, + { + "budget": 0.82, + "threshold": -0.6330366134643555, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.5852835757959097, + "rescued": 1639, + "harmed": 1287, + "missed_rescue": 950, + "avoided_harm": 154, + "selected_both_correct": 6203, + "selected_both_wrong": 2538 + }, + { + "budget": 0.83, + "threshold": -0.6429333686828613, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.584299669688664, + "rescued": 1655, + "harmed": 1317, + "missed_rescue": 934, + "avoided_harm": 124, + "selected_both_correct": 6256, + "selected_both_wrong": 2582 + }, + { + "budget": 0.84, + "threshold": -0.6513019800186157, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.5847213437346265, + "rescued": 1679, + "harmed": 1335, + "missed_rescue": 910, + "avoided_harm": 106, + "selected_both_correct": 6315, + "selected_both_wrong": 2623 + }, + { + "budget": 0.85, + "threshold": -0.6624053120613098, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.5849321807576077, + "rescued": 1702, + "harmed": 1355, + "missed_rescue": 887, + "avoided_harm": 86, + "selected_both_correct": 6367, + "selected_both_wrong": 2670 + }, + { + "budget": 0.86, + "threshold": -0.6759263277053833, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.584651064726966, + "rescued": 1724, + "harmed": 1381, + "missed_rescue": 865, + "avoided_harm": 60, + "selected_both_correct": 6420, + "selected_both_wrong": 2711 + }, + { + "budget": 0.87, + "threshold": -0.6904275417327881, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.5847213437346265, + "rescued": 1741, + "harmed": 1397, + "missed_rescue": 848, + "avoided_harm": 44, + "selected_both_correct": 6489, + "selected_both_wrong": 2752 + }, + { + "budget": 0.88, + "threshold": -0.7045450210571289, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.5859160868648534, + "rescued": 1778, + "harmed": 1417, + "missed_rescue": 811, + "avoided_harm": 24, + "selected_both_correct": 6525, + "selected_both_wrong": 2801 + }, + { + "budget": 0.89, + "threshold": -0.7205488681793213, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.5913275704547052, + "rescued": 1856, + "harmed": 1418, + "missed_rescue": 733, + "avoided_harm": 23, + "selected_both_correct": 6525, + "selected_both_wrong": 2864 + }, + { + "budget": 0.9, + "threshold": -0.7357243299484253, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.5963876590062548, + "rescued": 1928, + "harmed": 1418, + "missed_rescue": 661, + "avoided_harm": 23, + "selected_both_correct": 6525, + "selected_both_wrong": 2935 + }, + { + "budget": 0.91, + "threshold": -0.7513164281845093, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.6009557945041816, + "rescued": 1993, + "harmed": 1418, + "missed_rescue": 596, + "avoided_harm": 23, + "selected_both_correct": 6525, + "selected_both_wrong": 3012 + }, + { + "budget": 0.92, + "threshold": -0.7671180963516235, + "operator": "ge", + "actual_fraction": 0.9199522102747909, + "metric": 0.606156441071052, + "rescued": 2069, + "harmed": 1420, + "missed_rescue": 520, + "avoided_harm": 21, + "selected_both_correct": 6525, + "selected_both_wrong": 3076 + }, + { + "budget": 0.93, + "threshold": -0.7833515405654907, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.6112165296226018, + "rescued": 2142, + "harmed": 1421, + "missed_rescue": 447, + "avoided_harm": 20, + "selected_both_correct": 6526, + "selected_both_wrong": 3143 + }, + { + "budget": 0.94, + "threshold": -0.8013419508934021, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.6155738280975472, + "rescued": 2206, + "harmed": 1423, + "missed_rescue": 383, + "avoided_harm": 18, + "selected_both_correct": 6527, + "selected_both_wrong": 3219 + }, + { + "budget": 0.95, + "threshold": -0.8188987374305725, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.6207744746644177, + "rescued": 2280, + "harmed": 1423, + "missed_rescue": 309, + "avoided_harm": 18, + "selected_both_correct": 6528, + "selected_both_wrong": 3286 + }, + { + "budget": 0.96, + "threshold": -0.8395025134086609, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.6249912151240424, + "rescued": 2344, + "harmed": 1427, + "missed_rescue": 245, + "avoided_harm": 14, + "selected_both_correct": 6529, + "selected_both_wrong": 3359 + }, + { + "budget": 0.97, + "threshold": -0.8628664612770081, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.6291376765760067, + "rescued": 2405, + "harmed": 1429, + "missed_rescue": 184, + "avoided_harm": 12, + "selected_both_correct": 6531, + "selected_both_wrong": 3437 + }, + { + "budget": 0.98, + "threshold": -0.8857419490814209, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.6332138590203107, + "rescued": 2467, + "harmed": 1433, + "missed_rescue": 122, + "avoided_harm": 8, + "selected_both_correct": 6533, + "selected_both_wrong": 3511 + }, + { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.6372197624569541, + "rescued": 2529, + "harmed": 1438, + "missed_rescue": 60, + "avoided_harm": 3, + "selected_both_correct": 6534, + "selected_both_wrong": 3585 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.6412256658935975, + "rescued": 2589, + "harmed": 1441, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 6535, + "selected_both_wrong": 3664 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.26688029946402747, + "rescued": 0, + "harmed": 0, + "missed_rescue": 10723, + "avoided_harm": 2500, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.1584138125181198, + "operator": "ge", + "actual_fraction": 0.029095652667101495, + "metric": 0.2671071661514902, + "rescued": 14, + "harmed": 6, + "missed_rescue": 10709, + "avoided_harm": 2494, + "selected_both_correct": 996, + "selected_both_wrong": 10 + }, + { + "budget": 0.02, + "threshold": -0.21403978765010834, + "operator": "ge", + "actual_fraction": 0.04622408757054136, + "metric": 0.2674474661826844, + "rescued": 34, + "harmed": 14, + "missed_rescue": 10689, + "avoided_harm": 2486, + "selected_both_correct": 1522, + "selected_both_wrong": 60 + }, + { + "budget": 0.03, + "threshold": -0.24253316223621368, + "operator": "ge", + "actual_fraction": 0.056149505147037974, + "metric": 0.26801463290134137, + "rescued": 58, + "harmed": 18, + "missed_rescue": 10665, + "avoided_harm": 2482, + "selected_both_correct": 1796, + "selected_both_wrong": 108 + }, + { + "budget": 0.04, + "threshold": -0.2599165141582489, + "operator": "ge", + "actual_fraction": 0.06372118084110824, + "metric": 0.2684683662762669, + "rescued": 81, + "harmed": 25, + "missed_rescue": 10642, + "avoided_harm": 2475, + "selected_both_correct": 1988, + "selected_both_wrong": 153 + }, + { + "budget": 0.05, + "threshold": -0.2725968658924103, + "operator": "ge", + "actual_fraction": 0.06882568130902078, + "metric": 0.26883702464339393, + "rescued": 101, + "harmed": 32, + "missed_rescue": 10622, + "avoided_harm": 2468, + "selected_both_correct": 2095, + "selected_both_wrong": 199 + }, + { + "budget": 0.06, + "threshold": -0.2842792272567749, + "operator": "ge", + "actual_fraction": 0.07475257351898591, + "metric": 0.26920568301052095, + "rescued": 119, + "harmed": 37, + "missed_rescue": 10604, + "avoided_harm": 2463, + "selected_both_correct": 2209, + "selected_both_wrong": 271 + }, + { + "budget": 0.07, + "threshold": -0.2935555577278137, + "operator": "ge", + "actual_fraction": 0.08033916569775686, + "metric": 0.2694609080339166, + "rescued": 140, + "harmed": 49, + "missed_rescue": 10583, + "avoided_harm": 2451, + "selected_both_correct": 2306, + "selected_both_wrong": 338 + }, + { + "budget": 0.08, + "threshold": -0.30114513635635376, + "operator": "ge", + "actual_fraction": 0.08632277457958767, + "metric": 0.269942999744775, + "rescued": 170, + "harmed": 62, + "missed_rescue": 10553, + "avoided_harm": 2438, + "selected_both_correct": 2387, + "selected_both_wrong": 425 + }, + { + "budget": 0.09, + "threshold": -0.308601438999176, + "operator": "ge", + "actual_fraction": 0.09227802512548564, + "metric": 0.27028329977596915, + "rescued": 200, + "harmed": 80, + "missed_rescue": 10523, + "avoided_harm": 2420, + "selected_both_correct": 2473, + "selected_both_wrong": 501 + }, + { + "budget": 0.1, + "threshold": -0.3146189749240875, + "operator": "ge", + "actual_fraction": 0.09783625896832374, + "metric": 0.27076539148682754, + "rescued": 226, + "harmed": 89, + "missed_rescue": 10497, + "avoided_harm": 2411, + "selected_both_correct": 2547, + "selected_both_wrong": 588 + }, + { + "budget": 0.11, + "threshold": -0.32043084502220154, + "operator": "ge", + "actual_fraction": 0.10362135949862462, + "metric": 0.2713325582054845, + "rescued": 258, + "harmed": 101, + "missed_rescue": 10465, + "avoided_harm": 2399, + "selected_both_correct": 2611, + "selected_both_wrong": 684 + }, + { + "budget": 0.12, + "threshold": -0.32644838094711304, + "operator": "ge", + "actual_fraction": 0.11031392677877662, + "metric": 0.27221166661940277, + "rescued": 305, + "harmed": 117, + "missed_rescue": 10418, + "avoided_harm": 2383, + "selected_both_correct": 2671, + "selected_both_wrong": 797 + }, + { + "budget": 0.13, + "threshold": -0.33121857047080994, + "operator": "ge", + "actual_fraction": 0.11658111901993591, + "metric": 0.2733176417207838, + "rescued": 353, + "harmed": 126, + "missed_rescue": 10370, + "avoided_harm": 2374, + "selected_both_correct": 2743, + "selected_both_wrong": 889 + }, + { + "budget": 0.14, + "threshold": -0.33570271730422974, + "operator": "ge", + "actual_fraction": 0.12262144457363242, + "metric": 0.2742534668065678, + "rescued": 399, + "harmed": 139, + "missed_rescue": 10324, + "avoided_harm": 2361, + "selected_both_correct": 2797, + "selected_both_wrong": 989 + }, + { + "budget": 0.15, + "threshold": -0.3407592475414276, + "operator": "ge", + "actual_fraction": 0.13024983693956838, + "metric": 0.27535944190794887, + "rescued": 455, + "harmed": 156, + "missed_rescue": 10268, + "avoided_harm": 2344, + "selected_both_correct": 2866, + "selected_both_wrong": 1116 + }, + { + "budget": 0.16, + "threshold": -0.34456855058670044, + "operator": "ge", + "actual_fraction": 0.13714091257125033, + "metric": 0.2768907920483226, + "rescued": 526, + "harmed": 173, + "missed_rescue": 10197, + "avoided_harm": 2327, + "selected_both_correct": 2922, + "selected_both_wrong": 1215 + }, + { + "budget": 0.17, + "threshold": -0.3487326204776764, + "operator": "ge", + "actual_fraction": 0.143521538156141, + "metric": 0.27833706718089785, + "rescued": 589, + "harmed": 185, + "missed_rescue": 10134, + "avoided_harm": 2315, + "selected_both_correct": 2969, + "selected_both_wrong": 1318 + }, + { + "budget": 0.18, + "threshold": -0.35280776023864746, + "operator": "ge", + "actual_fraction": 0.15095142217054702, + "metric": 0.2800669256728015, + "rescued": 665, + "harmed": 200, + "missed_rescue": 10058, + "avoided_harm": 2300, + "selected_both_correct": 3012, + "selected_both_wrong": 1446 + }, + { + "budget": 0.19, + "threshold": -0.3564819097518921, + "operator": "ge", + "actual_fraction": 0.15823951450528884, + "metric": 0.28097439242265265, + "rescued": 720, + "harmed": 223, + "missed_rescue": 10003, + "avoided_harm": 2277, + "selected_both_correct": 3066, + "selected_both_wrong": 1571 + }, + { + "budget": 0.2, + "threshold": -0.35992002487182617, + "operator": "ge", + "actual_fraction": 0.16496044012137367, + "metric": 0.2826191759067578, + "rescued": 797, + "harmed": 242, + "missed_rescue": 9926, + "avoided_harm": 2258, + "selected_both_correct": 3110, + "selected_both_wrong": 1668 + }, + { + "budget": 0.21, + "threshold": -0.36340460181236267, + "operator": "ge", + "actual_fraction": 0.17290077418257097, + "metric": 0.28457590108612424, + "rescued": 880, + "harmed": 256, + "missed_rescue": 9843, + "avoided_harm": 2244, + "selected_both_correct": 3166, + "selected_both_wrong": 1795 + }, + { + "budget": 0.22, + "threshold": -0.3674520254135132, + "operator": "ge", + "actual_fraction": 0.1829679834387318, + "metric": 0.28667441794515497, + "rescued": 979, + "harmed": 281, + "missed_rescue": 9744, + "avoided_harm": 2219, + "selected_both_correct": 3230, + "selected_both_wrong": 1962 + }, + { + "budget": 0.23, + "threshold": -0.37108901143074036, + "operator": "ge", + "actual_fraction": 0.19122025919519042, + "metric": 0.28814905141366304, + "rescued": 1052, + "harmed": 302, + "missed_rescue": 9671, + "avoided_harm": 2198, + "selected_both_correct": 3292, + "selected_both_wrong": 2097 + }, + { + "budget": 0.24, + "threshold": -0.3745429813861847, + "operator": "ge", + "actual_fraction": 0.1999262683265746, + "metric": 0.28993562657743244, + "rescued": 1139, + "harmed": 326, + "missed_rescue": 9584, + "avoided_harm": 2174, + "selected_both_correct": 3343, + "selected_both_wrong": 2242 + }, + { + "budget": 0.25, + "threshold": -0.3773327171802521, + "operator": "ge", + "actual_fraction": 0.2080651107393018, + "metric": 0.2917789184130675, + "rescued": 1224, + "harmed": 346, + "missed_rescue": 9499, + "avoided_harm": 2154, + "selected_both_correct": 3394, + "selected_both_wrong": 2373 + }, + { + "budget": 0.26, + "threshold": -0.3803459703922272, + "operator": "ge", + "actual_fraction": 0.21603380313643195, + "metric": 0.29379236026429967, + "rescued": 1315, + "harmed": 366, + "missed_rescue": 9408, + "avoided_harm": 2134, + "selected_both_correct": 3435, + "selected_both_wrong": 2502 + }, + { + "budget": 0.27, + "threshold": -0.38361650705337524, + "operator": "ge", + "actual_fraction": 0.2243144372288234, + "metric": 0.29623117715452457, + "rescued": 1420, + "harmed": 385, + "missed_rescue": 9303, + "avoided_harm": 2115, + "selected_both_correct": 3485, + "selected_both_wrong": 2620 + }, + { + "budget": 0.28, + "threshold": -0.38652583956718445, + "operator": "ge", + "actual_fraction": 0.2324249213056178, + "metric": 0.29813118566202534, + "rescued": 1512, + "harmed": 410, + "missed_rescue": 9211, + "avoided_harm": 2090, + "selected_both_correct": 3520, + "selected_both_wrong": 2754 + }, + { + "budget": 0.29, + "threshold": -0.38952407240867615, + "operator": "ge", + "actual_fraction": 0.2419816805149874, + "metric": 0.3005983608881831, + "rescued": 1618, + "harmed": 429, + "missed_rescue": 9105, + "avoided_harm": 2071, + "selected_both_correct": 3587, + "selected_both_wrong": 2899 + }, + { + "budget": 0.3, + "threshold": -0.39262744784355164, + "operator": "ge", + "actual_fraction": 0.2512832147009613, + "metric": 0.3034625528174007, + "rescued": 1743, + "harmed": 453, + "missed_rescue": 8980, + "avoided_harm": 2047, + "selected_both_correct": 3641, + "selected_both_wrong": 3024 + }, + { + "budget": 0.31, + "threshold": -0.3954668939113617, + "operator": "ge", + "actual_fraction": 0.2596772821370842, + "metric": 0.30561778634829706, + "rescued": 1837, + "harmed": 471, + "missed_rescue": 8886, + "avoided_harm": 2029, + "selected_both_correct": 3675, + "selected_both_wrong": 3174 + }, + { + "budget": 0.32, + "threshold": -0.3986983895301819, + "operator": "ge", + "actual_fraction": 0.26985792473697645, + "metric": 0.308793919972776, + "rescued": 1971, + "harmed": 493, + "missed_rescue": 8752, + "avoided_harm": 2007, + "selected_both_correct": 3725, + "selected_both_wrong": 3327 + }, + { + "budget": 0.33, + "threshold": -0.4014112949371338, + "operator": "ge", + "actual_fraction": 0.27757139211071097, + "metric": 0.31109094518333663, + "rescued": 2067, + "harmed": 508, + "missed_rescue": 8656, + "avoided_harm": 1992, + "selected_both_correct": 3757, + "selected_both_wrong": 3456 + }, + { + "budget": 0.34, + "threshold": -0.4042409658432007, + "operator": "ge", + "actual_fraction": 0.28729830133567763, + "metric": 0.3132178203783002, + "rescued": 2174, + "harmed": 540, + "missed_rescue": 8549, + "avoided_harm": 1960, + "selected_both_correct": 3810, + "selected_both_wrong": 3607 + }, + { + "budget": 0.35, + "threshold": -0.40724706649780273, + "operator": "ge", + "actual_fraction": 0.29793267731049544, + "metric": 0.31659246235430905, + "rescued": 2314, + "harmed": 561, + "missed_rescue": 8409, + "avoided_harm": 1939, + "selected_both_correct": 3868, + "selected_both_wrong": 3763 + }, + { + "budget": 0.36, + "threshold": -0.40968337655067444, + "operator": "ge", + "actual_fraction": 0.3067237614496781, + "metric": 0.31928650426792954, + "rescued": 2427, + "harmed": 579, + "missed_rescue": 8296, + "avoided_harm": 1921, + "selected_both_correct": 3912, + "selected_both_wrong": 3898 + }, + { + "budget": 0.37, + "threshold": -0.4123799502849579, + "operator": "ge", + "actual_fraction": 0.3161387289793835, + "metric": 0.32192382950968434, + "rescued": 2547, + "harmed": 606, + "missed_rescue": 8176, + "avoided_harm": 1894, + "selected_both_correct": 3952, + "selected_both_wrong": 4043 + }, + { + "budget": 0.38, + "threshold": -0.41514647006988525, + "operator": "ge", + "actual_fraction": 0.32538354649349177, + "metric": 0.3242492130561779, + "rescued": 2654, + "harmed": 631, + "missed_rescue": 8069, + "avoided_harm": 1869, + "selected_both_correct": 4003, + "selected_both_wrong": 4186 + }, + { + "budget": 0.39, + "threshold": -0.4179389476776123, + "operator": "ge", + "actual_fraction": 0.3354507557496526, + "metric": 0.3270283299775969, + "rescued": 2782, + "harmed": 661, + "missed_rescue": 7941, + "avoided_harm": 1839, + "selected_both_correct": 4053, + "selected_both_wrong": 4333 + }, + { + "budget": 0.4, + "threshold": -0.42099592089653015, + "operator": "ge", + "actual_fraction": 0.34653886509939597, + "metric": 0.33009103025834446, + "rescued": 2921, + "harmed": 692, + "missed_rescue": 7802, + "avoided_harm": 1808, + "selected_both_correct": 4098, + "selected_both_wrong": 4509 + }, + { + "budget": 0.41, + "threshold": -0.4243226647377014, + "operator": "ge", + "actual_fraction": 0.3577404077928707, + "metric": 0.33306865553129344, + "rescued": 3055, + "harmed": 721, + "missed_rescue": 7668, + "avoided_harm": 1779, + "selected_both_correct": 4157, + "selected_both_wrong": 4682 + }, + { + "budget": 0.42, + "threshold": -0.4275209903717041, + "operator": "ge", + "actual_fraction": 0.3693673255253382, + "metric": 0.33661344752289935, + "rescued": 3203, + "harmed": 744, + "missed_rescue": 7520, + "avoided_harm": 1756, + "selected_both_correct": 4207, + "selected_both_wrong": 4871 + }, + { + "budget": 0.43, + "threshold": -0.4307159185409546, + "operator": "ge", + "actual_fraction": 0.38039871820321586, + "metric": 0.33953435612398264, + "rescued": 3331, + "harmed": 769, + "missed_rescue": 7392, + "avoided_harm": 1731, + "selected_both_correct": 4259, + "selected_both_wrong": 5055 + }, + { + "budget": 0.44, + "threshold": -0.4336964190006256, + "operator": "ge", + "actual_fraction": 0.39049428579530954, + "metric": 0.3427388480843944, + "rescued": 3465, + "harmed": 790, + "missed_rescue": 7258, + "avoided_harm": 1710, + "selected_both_correct": 4309, + "selected_both_wrong": 5206 + }, + { + "budget": 0.45, + "threshold": -0.4373021125793457, + "operator": "ge", + "actual_fraction": 0.4023764285511726, + "metric": 0.34597169838073905, + "rescued": 3606, + "harmed": 817, + "missed_rescue": 7117, + "avoided_harm": 1683, + "selected_both_correct": 4365, + "selected_both_wrong": 5401 + }, + { + "budget": 0.46, + "threshold": -0.4404309391975403, + "operator": "ge", + "actual_fraction": 0.41196154609647506, + "metric": 0.3488358903099566, + "rescued": 3729, + "harmed": 839, + "missed_rescue": 6994, + "avoided_harm": 1661, + "selected_both_correct": 4405, + "selected_both_wrong": 5554 + }, + { + "budget": 0.47, + "threshold": -0.44344601035118103, + "operator": "ge", + "actual_fraction": 0.42157502197771035, + "metric": 0.35172844057510705, + "rescued": 3850, + "harmed": 858, + "missed_rescue": 6873, + "avoided_harm": 1642, + "selected_both_correct": 4436, + "selected_both_wrong": 5722 + }, + { + "budget": 0.48, + "threshold": -0.4465356171131134, + "operator": "ge", + "actual_fraction": 0.43184073958540115, + "metric": 0.35459263250432466, + "rescued": 3979, + "harmed": 886, + "missed_rescue": 6744, + "avoided_harm": 1614, + "selected_both_correct": 4479, + "selected_both_wrong": 5884 + }, + { + "budget": 0.49, + "threshold": -0.44965147972106934, + "operator": "ge", + "actual_fraction": 0.44165272381816634, + "metric": 0.35751354110540795, + "rescued": 4106, + "harmed": 910, + "missed_rescue": 6617, + "avoided_harm": 1590, + "selected_both_correct": 4530, + "selected_both_wrong": 6028 + }, + { + "budget": 0.5, + "threshold": -0.453143447637558, + "operator": "ge", + "actual_fraction": 0.452315458128917, + "metric": 0.3607463914017525, + "rescued": 4242, + "harmed": 932, + "missed_rescue": 6481, + "avoided_harm": 1568, + "selected_both_correct": 4574, + "selected_both_wrong": 6202 + }, + { + "budget": 0.51, + "threshold": -0.4565357565879822, + "operator": "ge", + "actual_fraction": 0.4631483424552647, + "metric": 0.36440461673708985, + "rescued": 4392, + "harmed": 953, + "missed_rescue": 6331, + "avoided_harm": 1547, + "selected_both_correct": 4610, + "selected_both_wrong": 6377 + }, + { + "budget": 0.52, + "threshold": -0.45961302518844604, + "operator": "ge", + "actual_fraction": 0.47176927657885037, + "metric": 0.36641805858832205, + "rescued": 4484, + "harmed": 974, + "missed_rescue": 6239, + "avoided_harm": 1526, + "selected_both_correct": 4654, + "selected_both_wrong": 6524 + }, + { + "budget": 0.53, + "threshold": -0.46368208527565, + "operator": "ge", + "actual_fraction": 0.4840484360377733, + "metric": 0.36970762555653236, + "rescued": 4626, + "harmed": 1000, + "missed_rescue": 6097, + "avoided_harm": 1500, + "selected_both_correct": 4715, + "selected_both_wrong": 6728 + }, + { + "budget": 0.54, + "threshold": -0.46793216466903687, + "operator": "ge", + "actual_fraction": 0.495079828715651, + "metric": 0.3731389842044069, + "rescued": 4772, + "harmed": 1025, + "missed_rescue": 5951, + "avoided_harm": 1475, + "selected_both_correct": 4766, + "selected_both_wrong": 6895 + }, + { + "budget": 0.55, + "threshold": -0.4718930721282959, + "operator": "ge", + "actual_fraction": 0.5065365964325214, + "metric": 0.37634347616481867, + "rescued": 4911, + "harmed": 1051, + "missed_rescue": 5812, + "avoided_harm": 1449, + "selected_both_correct": 4826, + "selected_both_wrong": 7074 + }, + { + "budget": 0.56, + "threshold": -0.47629430890083313, + "operator": "ge", + "actual_fraction": 0.5183620225165188, + "metric": 0.379434534781499, + "rescued": 5056, + "harmed": 1087, + "missed_rescue": 5667, + "avoided_harm": 1413, + "selected_both_correct": 4873, + "selected_both_wrong": 7263 + }, + { + "budget": 0.57, + "threshold": -0.4799485504627228, + "operator": "ge", + "actual_fraction": 0.5273232566712985, + "metric": 0.3818733516717239, + "rescued": 5170, + "harmed": 1115, + "missed_rescue": 5553, + "avoided_harm": 1385, + "selected_both_correct": 4915, + "selected_both_wrong": 7395 + }, + { + "budget": 0.58, + "threshold": -0.4838995635509491, + "operator": "ge", + "actual_fraction": 0.5381561409976463, + "metric": 0.3852763519836656, + "rescued": 5317, + "harmed": 1142, + "missed_rescue": 5406, + "avoided_harm": 1358, + "selected_both_correct": 4957, + "selected_both_wrong": 7561 + }, + { + "budget": 0.59, + "threshold": -0.4886912405490875, + "operator": "ge", + "actual_fraction": 0.5503502254487707, + "metric": 0.3888211439752715, + "rescued": 5470, + "harmed": 1170, + "missed_rescue": 5253, + "avoided_harm": 1330, + "selected_both_correct": 4996, + "selected_both_wrong": 7771 + }, + { + "budget": 0.6, + "threshold": -0.49332454800605774, + "operator": "ge", + "actual_fraction": 0.5619487848453053, + "metric": 0.39256444431840737, + "rescued": 5631, + "harmed": 1199, + "missed_rescue": 5092, + "avoided_harm": 1301, + "selected_both_correct": 5037, + "selected_both_wrong": 7949 + }, + { + "budget": 0.61, + "threshold": -0.49750789999961853, + "operator": "ge", + "actual_fraction": 0.5722712191248618, + "metric": 0.3955987862632221, + "rescued": 5771, + "harmed": 1232, + "missed_rescue": 4952, + "avoided_harm": 1268, + "selected_both_correct": 5078, + "selected_both_wrong": 8099 + }, + { + "budget": 0.62, + "threshold": -0.5024909377098083, + "operator": "ge", + "actual_fraction": 0.5818563366701642, + "metric": 0.39840626152057396, + "rescued": 5893, + "harmed": 1255, + "missed_rescue": 4830, + "avoided_harm": 1245, + "selected_both_correct": 5118, + "selected_both_wrong": 8252 + }, + { + "budget": 0.63, + "threshold": -0.5067122578620911, + "operator": "ge", + "actual_fraction": 0.590817570824944, + "metric": 0.4011003034341945, + "rescued": 6008, + "harmed": 1275, + "missed_rescue": 4715, + "avoided_harm": 1225, + "selected_both_correct": 5160, + "selected_both_wrong": 8391 + }, + { + "budget": 0.64, + "threshold": -0.512323796749115, + "operator": "ge", + "actual_fraction": 0.604202705385248, + "metric": 0.40532569548818875, + "rescued": 6184, + "harmed": 1302, + "missed_rescue": 4539, + "avoided_harm": 1198, + "selected_both_correct": 5218, + "selected_both_wrong": 8602 + }, + { + "budget": 0.65, + "threshold": -0.5170255303382874, + "operator": "ge", + "actual_fraction": 0.6130505061962964, + "metric": 0.4078495873862122, + "rescued": 6296, + "harmed": 1325, + "missed_rescue": 4427, + "avoided_harm": 1175, + "selected_both_correct": 5258, + "selected_both_wrong": 8739 + }, + { + "budget": 0.66, + "threshold": -0.5215396881103516, + "operator": "ge", + "actual_fraction": 0.6222102487026061, + "metric": 0.4103167626123699, + "rescued": 6411, + "harmed": 1353, + "missed_rescue": 4312, + "avoided_harm": 1147, + "selected_both_correct": 5298, + "selected_both_wrong": 8879 + }, + { + "budget": 0.67, + "threshold": -0.5264327526092529, + "operator": "ge", + "actual_fraction": 0.6320505912713043, + "metric": 0.41216005444800496, + "rescued": 6514, + "harmed": 1391, + "missed_rescue": 4209, + "avoided_harm": 1109, + "selected_both_correct": 5333, + "selected_both_wrong": 9050 + }, + { + "budget": 0.68, + "threshold": -0.5312389135360718, + "operator": "ge", + "actual_fraction": 0.6424297422227264, + "metric": 0.4155630547599467, + "rescued": 6659, + "harmed": 1416, + "missed_rescue": 4064, + "avoided_harm": 1084, + "selected_both_correct": 5378, + "selected_both_wrong": 9201 + }, + { + "budget": 0.69, + "threshold": -0.5368650555610657, + "operator": "ge", + "actual_fraction": 0.6538865099395967, + "metric": 0.4177182882908431, + "rescued": 6784, + "harmed": 1465, + "missed_rescue": 3939, + "avoided_harm": 1035, + "selected_both_correct": 5424, + "selected_both_wrong": 9385 + }, + { + "budget": 0.7, + "threshold": -0.5431069135665894, + "operator": "ge", + "actual_fraction": 0.6647761109378102, + "metric": 0.4212347219465162, + "rescued": 6935, + "harmed": 1492, + "missed_rescue": 3788, + "avoided_harm": 1008, + "selected_both_correct": 5463, + "selected_both_wrong": 9552 + }, + { + "budget": 0.71, + "threshold": -0.5487136244773865, + "operator": "ge", + "actual_fraction": 0.6740492867878513, + "metric": 0.42384368885233814, + "rescued": 7054, + "harmed": 1519, + "missed_rescue": 3669, + "avoided_harm": 981, + "selected_both_correct": 5498, + "selected_both_wrong": 9698 + }, + { + "budget": 0.72, + "threshold": -0.5549414753913879, + "operator": "ge", + "actual_fraction": 0.6839179876924822, + "metric": 0.42628250574256304, + "rescued": 7167, + "harmed": 1546, + "missed_rescue": 3556, + "avoided_harm": 954, + "selected_both_correct": 5541, + "selected_both_wrong": 9863 + }, + { + "budget": 0.73, + "threshold": -0.5604677200317383, + "operator": "ge", + "actual_fraction": 0.6935314635737175, + "metric": 0.429260131015512, + "rescued": 7298, + "harmed": 1572, + "missed_rescue": 3425, + "avoided_harm": 928, + "selected_both_correct": 5579, + "selected_both_wrong": 10007 + }, + { + "budget": 0.74, + "threshold": -0.5666090250015259, + "operator": "ge", + "actual_fraction": 0.7039673311970054, + "metric": 0.4321243229447296, + "rescued": 7424, + "harmed": 1597, + "missed_rescue": 3299, + "avoided_harm": 903, + "selected_both_correct": 5614, + "selected_both_wrong": 10189 + }, + { + "budget": 0.75, + "threshold": -0.574927568435669, + "operator": "ge", + "actual_fraction": 0.7186853075461532, + "metric": 0.4360094149675297, + "rescued": 7607, + "harmed": 1643, + "missed_rescue": 3116, + "avoided_harm": 857, + "selected_both_correct": 5687, + "selected_both_wrong": 10406 + }, + { + "budget": 0.76, + "threshold": -0.5812664031982422, + "operator": "ge", + "actual_fraction": 0.7283555000992542, + "metric": 0.438363156849956, + "rescued": 7724, + "harmed": 1677, + "missed_rescue": 2999, + "avoided_harm": 823, + "selected_both_correct": 5729, + "selected_both_wrong": 10554 + }, + { + "budget": 0.77, + "threshold": -0.5883752107620239, + "operator": "ge", + "actual_fraction": 0.7390465927459376, + "metric": 0.44173779882596487, + "rescued": 7869, + "harmed": 1703, + "missed_rescue": 2854, + "avoided_harm": 797, + "selected_both_correct": 5774, + "selected_both_wrong": 10715 + }, + { + "budget": 0.78, + "threshold": -0.5972744226455688, + "operator": "ge", + "actual_fraction": 0.752176502282846, + "metric": 0.44596319087995917, + "rescued": 8052, + "harmed": 1737, + "missed_rescue": 2671, + "avoided_harm": 763, + "selected_both_correct": 5823, + "selected_both_wrong": 10912 + }, + { + "budget": 0.79, + "threshold": -0.6058458089828491, + "operator": "ge", + "actual_fraction": 0.7635765533278507, + "metric": 0.44888409948104246, + "rescued": 8191, + "harmed": 1773, + "missed_rescue": 2532, + "avoided_harm": 727, + "selected_both_correct": 5871, + "selected_both_wrong": 11091 + }, + { + "budget": 0.8, + "threshold": -0.6140543818473816, + "operator": "ge", + "actual_fraction": 0.7745228709979298, + "metric": 0.4523721748007827, + "rescued": 8338, + "harmed": 1797, + "missed_rescue": 2385, + "avoided_harm": 703, + "selected_both_correct": 5916, + "selected_both_wrong": 11261 + }, + { + "budget": 0.81, + "threshold": -0.6230268478393555, + "operator": "ge", + "actual_fraction": 0.7864050137537929, + "metric": 0.4548109916910076, + "rescued": 8468, + "harmed": 1841, + "missed_rescue": 2255, + "avoided_harm": 659, + "selected_both_correct": 5956, + "selected_both_wrong": 11466 + }, + { + "budget": 0.82, + "threshold": -0.6330366134643555, + "operator": "ge", + "actual_fraction": 0.798287156509656, + "metric": 0.45827070867481495, + "rescued": 8625, + "harmed": 1876, + "missed_rescue": 2098, + "avoided_harm": 624, + "selected_both_correct": 5991, + "selected_both_wrong": 11658 + }, + { + "budget": 0.83, + "threshold": -0.6429333686828613, + "operator": "ge", + "actual_fraction": 0.8092334741797351, + "metric": 0.4612766922836968, + "rescued": 8762, + "harmed": 1907, + "missed_rescue": 1961, + "avoided_harm": 593, + "selected_both_correct": 6039, + "selected_both_wrong": 11828 + }, + { + "budget": 0.84, + "threshold": -0.6513019800186157, + "operator": "ge", + "actual_fraction": 0.8177126166236566, + "metric": 0.46340356747866035, + "rescued": 8858, + "harmed": 1928, + "missed_rescue": 1865, + "avoided_harm": 572, + "selected_both_correct": 6071, + "selected_both_wrong": 11978 + }, + { + "budget": 0.85, + "threshold": -0.6624053120613098, + "operator": "ge", + "actual_fraction": 0.8288858009811985, + "metric": 0.4659274593766838, + "rescued": 8984, + "harmed": 1965, + "missed_rescue": 1739, + "avoided_harm": 535, + "selected_both_correct": 6119, + "selected_both_wrong": 12161 + }, + { + "budget": 0.86, + "threshold": -0.6759263277053833, + "operator": "ge", + "actual_fraction": 0.8421291438618382, + "metric": 0.4693021013526926, + "rescued": 9143, + "harmed": 2005, + "missed_rescue": 1580, + "avoided_harm": 495, + "selected_both_correct": 6163, + "selected_both_wrong": 12385 + }, + { + "budget": 0.87, + "threshold": -0.6904275417327881, + "operator": "ge", + "actual_fraction": 0.8543799449848283, + "metric": 0.47202450160224596, + "rescued": 9273, + "harmed": 2039, + "missed_rescue": 1450, + "avoided_harm": 461, + "selected_both_correct": 6209, + "selected_both_wrong": 12607 + }, + { + "budget": 0.88, + "threshold": -0.7045450210571289, + "operator": "ge", + "actual_fraction": 0.8657516376939001, + "metric": 0.47395286844567963, + "rescued": 9384, + "harmed": 2082, + "missed_rescue": 1339, + "avoided_harm": 418, + "selected_both_correct": 6264, + "selected_both_wrong": 12799 + }, + { + "budget": 0.89, + "threshold": -0.7205488681793213, + "operator": "ge", + "actual_fraction": 0.8767830303717777, + "metric": 0.4759663102969118, + "rescued": 9501, + "harmed": 2128, + "missed_rescue": 1222, + "avoided_harm": 372, + "selected_both_correct": 6312, + "selected_both_wrong": 12977 + }, + { + "budget": 0.9, + "threshold": -0.7357243299484253, + "operator": "ge", + "actual_fraction": 0.8868785979638715, + "metric": 0.4782066188356067, + "rescued": 9610, + "harmed": 2158, + "missed_rescue": 1113, + "avoided_harm": 342, + "selected_both_correct": 6365, + "selected_both_wrong": 13141 + }, + { + "budget": 0.91, + "threshold": -0.7513164281845093, + "operator": "ge", + "actual_fraction": 0.8974562572668235, + "metric": 0.4802200606868389, + "rescued": 9710, + "harmed": 2187, + "missed_rescue": 1013, + "avoided_harm": 313, + "selected_both_correct": 6424, + "selected_both_wrong": 13326 + }, + { + "budget": 0.92, + "threshold": -0.7671180963516235, + "operator": "ge", + "actual_fraction": 0.9084025749369027, + "metric": 0.4826021609051981, + "rescued": 9825, + "harmed": 2218, + "missed_rescue": 898, + "avoided_harm": 282, + "selected_both_correct": 6474, + "selected_both_wrong": 13516 + }, + { + "budget": 0.93, + "threshold": -0.7833515405654907, + "operator": "ge", + "actual_fraction": 0.9181011258259365, + "metric": 0.4839917193659076, + "rescued": 9916, + "harmed": 2260, + "missed_rescue": 807, + "avoided_harm": 240, + "selected_both_correct": 6514, + "selected_both_wrong": 13685 + }, + { + "budget": 0.94, + "threshold": -0.8013419508934021, + "operator": "ge", + "actual_fraction": 0.9289623684882171, + "metric": 0.48634546124833394, + "rescued": 10031, + "harmed": 2292, + "missed_rescue": 692, + "avoided_harm": 208, + "selected_both_correct": 6566, + "selected_both_wrong": 13869 + }, + { + "budget": 0.95, + "threshold": -0.8188987374305725, + "operator": "ge", + "actual_fraction": 0.9394549527833707, + "metric": 0.48770666137311064, + "rescued": 10126, + "harmed": 2339, + "missed_rescue": 597, + "avoided_harm": 161, + "selected_both_correct": 6606, + "selected_both_wrong": 14057 + }, + { + "budget": 0.96, + "threshold": -0.8395025134086609, + "operator": "ge", + "actual_fraction": 0.9509684371721068, + "metric": 0.4894648782009472, + "rescued": 10222, + "harmed": 2373, + "missed_rescue": 501, + "avoided_harm": 127, + "selected_both_correct": 6654, + "selected_both_wrong": 14285 + }, + { + "budget": 0.97, + "threshold": -0.8628664612770081, + "operator": "ge", + "actual_fraction": 0.9640699883730822, + "metric": 0.49218727845050053, + "rescued": 10358, + "harmed": 2413, + "missed_rescue": 365, + "avoided_harm": 87, + "selected_both_correct": 6713, + "selected_both_wrong": 14512 + }, + { + "budget": 0.98, + "threshold": -0.8857419490814209, + "operator": "ge", + "actual_fraction": 0.9760372061367439, + "metric": 0.49439922865326263, + "rescued": 10476, + "harmed": 2453, + "missed_rescue": 247, + "avoided_harm": 47, + "selected_both_correct": 6784, + "selected_both_wrong": 14705 + }, + { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9873805405098829, + "metric": 0.4968380455434875, + "rescued": 10590, + "harmed": 2481, + "missed_rescue": 133, + "avoided_harm": 19, + "selected_both_correct": 6845, + "selected_both_wrong": 14902 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5000708958398321, + "rescued": 10723, + "harmed": 2500, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 6911, + "selected_both_wrong": 15129 + } + ] + }, + "learned": { + "selected_budget_index": 34, + "validation_area_0_25": 0.6255183076814955, + "validation_point": { + "budget": 0.34, + "threshold": 6.076109886169434, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.6800899571298054, + "rescued": 1909, + "harmed": 208, + "missed_rescue": 680, + "avoided_harm": 1233, + "selected_both_correct": 269, + "selected_both_wrong": 2451 + }, + "test_point": { + "budget": 0.34, + "threshold": 6.076109886169434, + "operator": "ge", + "actual_fraction": 0.43518702322547714, + "metric": 0.45622890848764996, + "rescued": 7140, + "harmed": 463, + "missed_rescue": 3583, + "avoided_harm": 2037, + "selected_both_correct": 329, + "selected_both_wrong": 7414 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.5605453650994447, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2589, + "avoided_harm": 1441, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 15.251158714294434, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.567924660903788, + "rescued": 105, + "harmed": 0, + "missed_rescue": 2484, + "avoided_harm": 1441, + "selected_both_correct": 0, + "selected_both_wrong": 37 + }, + { + "budget": 0.02, + "threshold": 14.266501426696777, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.5753742357157917, + "rescued": 211, + "harmed": 0, + "missed_rescue": 2378, + "avoided_harm": 1441, + "selected_both_correct": 0, + "selected_both_wrong": 73 + }, + { + "budget": 0.03, + "threshold": 13.713798522949219, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.5814182303745871, + "rescued": 298, + "harmed": 1, + "missed_rescue": 2291, + "avoided_harm": 1440, + "selected_both_correct": 1, + "selected_both_wrong": 126 + }, + { + "budget": 0.04, + "threshold": 13.269920349121094, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.5877433410640241, + "rescued": 390, + "harmed": 3, + "missed_rescue": 2199, + "avoided_harm": 1438, + "selected_both_correct": 1, + "selected_both_wrong": 175 + }, + { + "budget": 0.05, + "threshold": 12.880468368530273, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.5940684517534612, + "rescued": 481, + "harmed": 4, + "missed_rescue": 2108, + "avoided_harm": 1437, + "selected_both_correct": 1, + "selected_both_wrong": 225 + }, + { + "budget": 0.06, + "threshold": 12.571352005004883, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.5999016093892754, + "rescued": 565, + "harmed": 5, + "missed_rescue": 2024, + "avoided_harm": 1436, + "selected_both_correct": 3, + "selected_both_wrong": 280 + }, + { + "budget": 0.07, + "threshold": 12.294123649597168, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.6048914189331647, + "rescued": 636, + "harmed": 5, + "missed_rescue": 1953, + "avoided_harm": 1436, + "selected_both_correct": 4, + "selected_both_wrong": 351 + }, + { + "budget": 0.08, + "threshold": 12.034196853637695, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.6096703914540726, + "rescued": 704, + "harmed": 5, + "missed_rescue": 1885, + "avoided_harm": 1436, + "selected_both_correct": 5, + "selected_both_wrong": 424 + }, + { + "budget": 0.09, + "threshold": 11.747140884399414, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.614168247944339, + "rescued": 770, + "harmed": 7, + "missed_rescue": 1819, + "avoided_harm": 1434, + "selected_both_correct": 5, + "selected_both_wrong": 498 + }, + { + "budget": 0.1, + "threshold": 11.491235733032227, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.6191580574882283, + "rescued": 841, + "harmed": 7, + "missed_rescue": 1748, + "avoided_harm": 1434, + "selected_both_correct": 6, + "selected_both_wrong": 568 + }, + { + "budget": 0.11, + "threshold": 11.249296188354492, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.6236559139784946, + "rescued": 908, + "harmed": 10, + "missed_rescue": 1681, + "avoided_harm": 1431, + "selected_both_correct": 10, + "selected_both_wrong": 637 + }, + { + "budget": 0.12, + "threshold": 11.011529922485352, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.6282943284840818, + "rescued": 975, + "harmed": 11, + "missed_rescue": 1614, + "avoided_harm": 1430, + "selected_both_correct": 11, + "selected_both_wrong": 710 + }, + { + "budget": 0.13, + "threshold": 10.77489185333252, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.6330030219973294, + "rescued": 1043, + "harmed": 12, + "missed_rescue": 1546, + "avoided_harm": 1429, + "selected_both_correct": 12, + "selected_both_wrong": 782 + }, + { + "budget": 0.14, + "threshold": 10.548977851867676, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.637711715510577, + "rescued": 1110, + "harmed": 12, + "missed_rescue": 1479, + "avoided_harm": 1429, + "selected_both_correct": 12, + "selected_both_wrong": 858 + }, + { + "budget": 0.15, + "threshold": 10.36727523803711, + "operator": "gt", + "actual_fraction": 0.14990512333965844, + "metric": 0.6412256658935975, + "rescued": 1162, + "harmed": 14, + "missed_rescue": 1427, + "avoided_harm": 1427, + "selected_both_correct": 12, + "selected_both_wrong": 945 + }, + { + "budget": 0.16, + "threshold": 10.15395736694336, + "operator": "ge", + "actual_fraction": 0.15995502143509735, + "metric": 0.644880174291939, + "rescued": 1215, + "harmed": 15, + "missed_rescue": 1374, + "avoided_harm": 1426, + "selected_both_correct": 16, + "selected_both_wrong": 1030 + }, + { + "budget": 0.17, + "threshold": 9.928215026855469, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.6486049616979408, + "rescued": 1272, + "harmed": 19, + "missed_rescue": 1317, + "avoided_harm": 1422, + "selected_both_correct": 18, + "selected_both_wrong": 1109 + }, + { + "budget": 0.18, + "threshold": 9.721980094909668, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.652048633073301, + "rescued": 1326, + "harmed": 24, + "missed_rescue": 1263, + "avoided_harm": 1417, + "selected_both_correct": 23, + "selected_both_wrong": 1188 + }, + { + "budget": 0.19, + "threshold": 9.500179290771484, + "operator": "gt", + "actual_fraction": 0.18989387869843277, + "metric": 0.6555625834563216, + "rescued": 1380, + "harmed": 28, + "missed_rescue": 1209, + "avoided_harm": 1413, + "selected_both_correct": 29, + "selected_both_wrong": 1265 + }, + { + "budget": 0.2, + "threshold": 9.266512870788574, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.6577412326937944, + "rescued": 1418, + "harmed": 35, + "missed_rescue": 1171, + "avoided_harm": 1406, + "selected_both_correct": 35, + "selected_both_wrong": 1357 + }, + { + "budget": 0.21, + "threshold": 9.037958145141602, + "operator": "gt", + "actual_fraction": 0.20992339588165015, + "metric": 0.660763230023192, + "rescued": 1463, + "harmed": 37, + "missed_rescue": 1126, + "avoided_harm": 1404, + "selected_both_correct": 40, + "selected_both_wrong": 1447 + }, + { + "budget": 0.22, + "threshold": 8.772733688354492, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.6638555063602501, + "rescued": 1511, + "harmed": 41, + "missed_rescue": 1078, + "avoided_harm": 1400, + "selected_both_correct": 47, + "selected_both_wrong": 1531 + }, + { + "budget": 0.23, + "threshold": 8.523207664489746, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.6667369456743271, + "rescued": 1558, + "harmed": 47, + "missed_rescue": 1031, + "avoided_harm": 1394, + "selected_both_correct": 56, + "selected_both_wrong": 1611 + }, + { + "budget": 0.24, + "threshold": 8.271183967590332, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.6686344788811582, + "rescued": 1597, + "harmed": 59, + "missed_rescue": 992, + "avoided_harm": 1382, + "selected_both_correct": 60, + "selected_both_wrong": 1698 + }, + { + "budget": 0.25, + "threshold": 7.994204521179199, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.6702508960573477, + "rescued": 1630, + "harmed": 69, + "missed_rescue": 959, + "avoided_harm": 1372, + "selected_both_correct": 69, + "selected_both_wrong": 1789 + }, + { + "budget": 0.26, + "threshold": 7.755735397338867, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.6720781502565184, + "rescued": 1668, + "harmed": 81, + "missed_rescue": 921, + "avoided_harm": 1360, + "selected_both_correct": 80, + "selected_both_wrong": 1870 + }, + { + "budget": 0.27, + "threshold": 7.53363561630249, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.674397357509312, + "rescued": 1712, + "harmed": 92, + "missed_rescue": 877, + "avoided_harm": 1349, + "selected_both_correct": 86, + "selected_both_wrong": 1951 + }, + { + "budget": 0.28, + "threshold": 7.3391337394714355, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.675240705601237, + "rescued": 1739, + "harmed": 107, + "missed_rescue": 850, + "avoided_harm": 1334, + "selected_both_correct": 106, + "selected_both_wrong": 2032 + }, + { + "budget": 0.29, + "threshold": 7.089168548583984, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.6765760067467848, + "rescued": 1773, + "harmed": 122, + "missed_rescue": 816, + "avoided_harm": 1319, + "selected_both_correct": 124, + "selected_both_wrong": 2107 + }, + { + "budget": 0.3, + "threshold": 6.880478382110596, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.6777707498770117, + "rescued": 1806, + "harmed": 138, + "missed_rescue": 783, + "avoided_harm": 1303, + "selected_both_correct": 148, + "selected_both_wrong": 2176 + }, + { + "budget": 0.31, + "threshold": 6.666043758392334, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.6788249349919179, + "rescued": 1835, + "harmed": 152, + "missed_rescue": 754, + "avoided_harm": 1289, + "selected_both_correct": 181, + "selected_both_wrong": 2242 + }, + { + "budget": 0.32, + "threshold": 6.450697898864746, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.6794574460608617, + "rescued": 1857, + "harmed": 165, + "missed_rescue": 732, + "avoided_harm": 1276, + "selected_both_correct": 210, + "selected_both_wrong": 2321 + }, + { + "budget": 0.33, + "threshold": 6.251960754394531, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.6797385620915033, + "rescued": 1882, + "harmed": 186, + "missed_rescue": 707, + "avoided_harm": 1255, + "selected_both_correct": 237, + "selected_both_wrong": 2390 + }, + { + "budget": 0.34, + "threshold": 6.076109886169434, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.6800899571298054, + "rescued": 1909, + "harmed": 208, + "missed_rescue": 680, + "avoided_harm": 1233, + "selected_both_correct": 269, + "selected_both_wrong": 2451 + }, + { + "budget": 0.35, + "threshold": 5.836214065551758, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.6800196781221449, + "rescued": 1939, + "harmed": 239, + "missed_rescue": 650, + "avoided_harm": 1202, + "selected_both_correct": 308, + "selected_both_wrong": 2494 + }, + { + "budget": 0.36, + "threshold": 5.620694637298584, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.6796682830838429, + "rescued": 1961, + "harmed": 266, + "missed_rescue": 628, + "avoided_harm": 1175, + "selected_both_correct": 355, + "selected_both_wrong": 2540 + }, + { + "budget": 0.37, + "threshold": 5.452563285827637, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.6791763300302199, + "rescued": 1977, + "harmed": 289, + "missed_rescue": 612, + "avoided_harm": 1152, + "selected_both_correct": 405, + "selected_both_wrong": 2593 + }, + { + "budget": 0.38, + "threshold": 5.261322021484375, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.678332981938295, + "rescued": 1993, + "harmed": 317, + "missed_rescue": 596, + "avoided_harm": 1124, + "selected_both_correct": 458, + "selected_both_wrong": 2639 + }, + { + "budget": 0.39, + "threshold": 5.070874214172363, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.6781221449153138, + "rescued": 2009, + "harmed": 336, + "missed_rescue": 580, + "avoided_harm": 1105, + "selected_both_correct": 509, + "selected_both_wrong": 2695 + }, + { + "budget": 0.4, + "threshold": 4.91162109375, + "operator": "gt", + "actual_fraction": 0.39988755358774336, + "metric": 0.677138238808068, + "rescued": 2020, + "harmed": 361, + "missed_rescue": 569, + "avoided_harm": 1080, + "selected_both_correct": 565, + "selected_both_wrong": 2744 + }, + { + "budget": 0.41, + "threshold": 4.7608537673950195, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.6761543327008223, + "rescued": 2033, + "harmed": 388, + "missed_rescue": 556, + "avoided_harm": 1053, + "selected_both_correct": 629, + "selected_both_wrong": 2783 + }, + { + "budget": 0.42, + "threshold": 4.58115816116333, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.6748893105629349, + "rescued": 2046, + "harmed": 419, + "missed_rescue": 543, + "avoided_harm": 1022, + "selected_both_correct": 690, + "selected_both_wrong": 2821 + }, + { + "budget": 0.43, + "threshold": 4.429701805114746, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.6735540094173871, + "rescued": 2056, + "harmed": 448, + "missed_rescue": 533, + "avoided_harm": 993, + "selected_both_correct": 751, + "selected_both_wrong": 2863 + }, + { + "budget": 0.44, + "threshold": 4.268267631530762, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.6725701033101412, + "rescued": 2069, + "harmed": 475, + "missed_rescue": 520, + "avoided_harm": 966, + "selected_both_correct": 811, + "selected_both_wrong": 2905 + }, + { + "budget": 0.45, + "threshold": 4.120080471038818, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.671515918195235, + "rescued": 2080, + "harmed": 501, + "missed_rescue": 509, + "avoided_harm": 940, + "selected_both_correct": 876, + "selected_both_wrong": 2946 + }, + { + "budget": 0.46, + "threshold": 3.951977014541626, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.6701806170496872, + "rescued": 2094, + "harmed": 534, + "missed_rescue": 495, + "avoided_harm": 907, + "selected_both_correct": 943, + "selected_both_wrong": 2974 + }, + { + "budget": 0.47, + "threshold": 3.798562526702881, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.6694778269730831, + "rescued": 2110, + "harmed": 560, + "missed_rescue": 479, + "avoided_harm": 881, + "selected_both_correct": 1008, + "selected_both_wrong": 3009 + }, + { + "budget": 0.48, + "threshold": 3.635770797729492, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.6667369456743271, + "rescued": 2116, + "harmed": 605, + "missed_rescue": 473, + "avoided_harm": 836, + "selected_both_correct": 1072, + "selected_both_wrong": 3036 + }, + { + "budget": 0.49, + "threshold": 3.44822359085083, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.6647691334598356, + "rescued": 2126, + "harmed": 643, + "missed_rescue": 463, + "avoided_harm": 798, + "selected_both_correct": 1144, + "selected_both_wrong": 3059 + }, + { + "budget": 0.5, + "threshold": 3.267157554626465, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.6635743903296085, + "rescued": 2138, + "harmed": 672, + "missed_rescue": 451, + "avoided_harm": 769, + "selected_both_correct": 1220, + "selected_both_wrong": 3084 + }, + { + "budget": 0.51, + "threshold": 3.117006301879883, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.6618174151380982, + "rescued": 2151, + "harmed": 710, + "missed_rescue": 438, + "avoided_harm": 731, + "selected_both_correct": 1289, + "selected_both_wrong": 3106 + }, + { + "budget": 0.52, + "threshold": 2.9639105796813965, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.6611849040691545, + "rescued": 2166, + "harmed": 734, + "missed_rescue": 423, + "avoided_harm": 707, + "selected_both_correct": 1373, + "selected_both_wrong": 3126 + }, + { + "budget": 0.53, + "threshold": 2.8179378509521484, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.6602712769695692, + "rescued": 2182, + "harmed": 763, + "missed_rescue": 407, + "avoided_harm": 678, + "selected_both_correct": 1451, + "selected_both_wrong": 3145 + }, + { + "budget": 0.54, + "threshold": 2.663231372833252, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.6594982078853047, + "rescued": 2199, + "harmed": 791, + "missed_rescue": 390, + "avoided_harm": 650, + "selected_both_correct": 1532, + "selected_both_wrong": 3161 + }, + { + "budget": 0.55, + "threshold": 2.5109760761260986, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.6572492796401714, + "rescued": 2209, + "harmed": 833, + "missed_rescue": 380, + "avoided_harm": 608, + "selected_both_correct": 1601, + "selected_both_wrong": 3182 + }, + { + "budget": 0.56, + "threshold": 2.3562464714050293, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.656476210555907, + "rescued": 2216, + "harmed": 851, + "missed_rescue": 373, + "avoided_harm": 590, + "selected_both_correct": 1696, + "selected_both_wrong": 3205 + }, + { + "budget": 0.57, + "threshold": 2.2040421962738037, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.6547192353643967, + "rescued": 2226, + "harmed": 886, + "missed_rescue": 363, + "avoided_harm": 555, + "selected_both_correct": 1774, + "selected_both_wrong": 3224 + }, + { + "budget": 0.58, + "threshold": 2.0592546463012695, + "operator": "ge", + "actual_fraction": 0.5799423712137185, + "metric": 0.6533136552111884, + "rescued": 2236, + "harmed": 916, + "missed_rescue": 353, + "avoided_harm": 525, + "selected_both_correct": 1864, + "selected_both_wrong": 3236 + }, + { + "budget": 0.59, + "threshold": 1.9159198999404907, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.6519783540656406, + "rescued": 2246, + "harmed": 945, + "missed_rescue": 343, + "avoided_harm": 496, + "selected_both_correct": 1948, + "selected_both_wrong": 3256 + }, + { + "budget": 0.6, + "threshold": 1.7584953308105469, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.6502213788741303, + "rescued": 2251, + "harmed": 975, + "missed_rescue": 338, + "avoided_harm": 466, + "selected_both_correct": 2038, + "selected_both_wrong": 3273 + }, + { + "budget": 0.61, + "threshold": 1.5953938961029053, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.6493077517745449, + "rescued": 2260, + "harmed": 997, + "missed_rescue": 329, + "avoided_harm": 444, + "selected_both_correct": 2133, + "selected_both_wrong": 3289 + }, + { + "budget": 0.62, + "threshold": 1.4405584335327148, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.6486049616979408, + "rescued": 2271, + "harmed": 1018, + "missed_rescue": 318, + "avoided_harm": 423, + "selected_both_correct": 2228, + "selected_both_wrong": 3304 + }, + { + "budget": 0.63, + "threshold": 1.296458125114441, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.6479021716213367, + "rescued": 2285, + "harmed": 1042, + "missed_rescue": 304, + "avoided_harm": 399, + "selected_both_correct": 2323, + "selected_both_wrong": 3314 + }, + { + "budget": 0.64, + "threshold": 1.132359266281128, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.6475507765830346, + "rescued": 2299, + "harmed": 1061, + "missed_rescue": 290, + "avoided_harm": 380, + "selected_both_correct": 2414, + "selected_both_wrong": 3332 + }, + { + "budget": 0.65, + "threshold": 0.9609370827674866, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.6467777074987702, + "rescued": 2315, + "harmed": 1088, + "missed_rescue": 274, + "avoided_harm": 353, + "selected_both_correct": 2493, + "selected_both_wrong": 3352 + }, + { + "budget": 0.66, + "threshold": 0.797613799571991, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.6456532433762036, + "rescued": 2325, + "harmed": 1114, + "missed_rescue": 264, + "avoided_harm": 327, + "selected_both_correct": 2586, + "selected_both_wrong": 3366 + }, + { + "budget": 0.67, + "threshold": 0.6462197303771973, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.6449504532995994, + "rescued": 2341, + "harmed": 1140, + "missed_rescue": 248, + "avoided_harm": 301, + "selected_both_correct": 2676, + "selected_both_wrong": 3376 + }, + { + "budget": 0.68, + "threshold": 0.49411773681640625, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.6444585002459765, + "rescued": 2351, + "harmed": 1157, + "missed_rescue": 238, + "avoided_harm": 284, + "selected_both_correct": 2781, + "selected_both_wrong": 3386 + }, + { + "budget": 0.69, + "threshold": 0.3384990692138672, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.6441071052076744, + "rescued": 2366, + "harmed": 1177, + "missed_rescue": 223, + "avoided_harm": 264, + "selected_both_correct": 2874, + "selected_both_wrong": 3401 + }, + { + "budget": 0.7, + "threshold": 0.1719987392425537, + "operator": "ge", + "actual_fraction": 0.6999789162977019, + "metric": 0.6438962681846933, + "rescued": 2378, + "harmed": 1192, + "missed_rescue": 211, + "avoided_harm": 249, + "selected_both_correct": 2970, + "selected_both_wrong": 3420 + }, + { + "budget": 0.71, + "threshold": 0.0025517642498016357, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.6434043151310703, + "rescued": 2386, + "harmed": 1207, + "missed_rescue": 203, + "avoided_harm": 234, + "selected_both_correct": 3074, + "selected_both_wrong": 3435 + }, + { + "budget": 0.72, + "threshold": -0.1762939691543579, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.6431231991004287, + "rescued": 2398, + "harmed": 1223, + "missed_rescue": 191, + "avoided_harm": 218, + "selected_both_correct": 3175, + "selected_both_wrong": 3448 + }, + { + "budget": 0.73, + "threshold": -0.3592407703399658, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.6422798510085037, + "rescued": 2405, + "harmed": 1242, + "missed_rescue": 184, + "avoided_harm": 199, + "selected_both_correct": 3281, + "selected_both_wrong": 3459 + }, + { + "budget": 0.74, + "threshold": -0.5128207206726074, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.6427015250544662, + "rescued": 2424, + "harmed": 1255, + "missed_rescue": 165, + "avoided_harm": 186, + "selected_both_correct": 3378, + "selected_both_wrong": 3472 + }, + { + "budget": 0.75, + "threshold": -0.6840166449546814, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.6430529200927683, + "rescued": 2441, + "harmed": 1267, + "missed_rescue": 148, + "avoided_harm": 174, + "selected_both_correct": 3481, + "selected_both_wrong": 3482 + }, + { + "budget": 0.76, + "threshold": -0.8224172592163086, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.6434745941387308, + "rescued": 2451, + "harmed": 1271, + "missed_rescue": 138, + "avoided_harm": 170, + "selected_both_correct": 3601, + "selected_both_wrong": 3491 + }, + { + "budget": 0.77, + "threshold": -0.9985247850418091, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.6431231991004287, + "rescued": 2459, + "harmed": 1284, + "missed_rescue": 130, + "avoided_harm": 157, + "selected_both_correct": 3712, + "selected_both_wrong": 3501 + }, + { + "budget": 0.78, + "threshold": -1.168760895729065, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.6431231991004287, + "rescued": 2474, + "harmed": 1299, + "missed_rescue": 115, + "avoided_harm": 142, + "selected_both_correct": 3811, + "selected_both_wrong": 3514 + }, + { + "budget": 0.79, + "threshold": -1.3431593179702759, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.6434745941387308, + "rescued": 2487, + "harmed": 1307, + "missed_rescue": 102, + "avoided_harm": 134, + "selected_both_correct": 3922, + "selected_both_wrong": 3524 + }, + { + "budget": 0.8, + "threshold": -1.5382206439971924, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.6435448731463912, + "rescued": 2500, + "harmed": 1319, + "missed_rescue": 89, + "avoided_harm": 122, + "selected_both_correct": 4031, + "selected_both_wrong": 3533 + }, + { + "budget": 0.81, + "threshold": -1.7282600402832031, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.6438259891770328, + "rescued": 2510, + "harmed": 1325, + "missed_rescue": 79, + "avoided_harm": 116, + "selected_both_correct": 4141, + "selected_both_wrong": 3549 + }, + { + "budget": 0.82, + "threshold": -1.8991161584854126, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.6441773842153349, + "rescued": 2522, + "harmed": 1332, + "missed_rescue": 67, + "avoided_harm": 109, + "selected_both_correct": 4249, + "selected_both_wrong": 3564 + }, + { + "budget": 0.83, + "threshold": -2.085026979446411, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.6442476632229953, + "rescued": 2528, + "harmed": 1337, + "missed_rescue": 61, + "avoided_harm": 104, + "selected_both_correct": 4366, + "selected_both_wrong": 3579 + }, + { + "budget": 0.84, + "threshold": -2.285501003265381, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.6441773842153349, + "rescued": 2535, + "harmed": 1345, + "missed_rescue": 54, + "avoided_harm": 96, + "selected_both_correct": 4482, + "selected_both_wrong": 3590 + }, + { + "budget": 0.85, + "threshold": -2.495553970336914, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.6438259891770328, + "rescued": 2541, + "harmed": 1356, + "missed_rescue": 48, + "avoided_harm": 85, + "selected_both_correct": 4595, + "selected_both_wrong": 3602 + }, + { + "budget": 0.86, + "threshold": -2.6960723400115967, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.6441773842153349, + "rescued": 2551, + "harmed": 1361, + "missed_rescue": 38, + "avoided_harm": 80, + "selected_both_correct": 4710, + "selected_both_wrong": 3614 + }, + { + "budget": 0.87, + "threshold": -2.9289634227752686, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.6438259891770328, + "rescued": 2555, + "harmed": 1370, + "missed_rescue": 34, + "avoided_harm": 71, + "selected_both_correct": 4829, + "selected_both_wrong": 3625 + }, + { + "budget": 0.88, + "threshold": -3.1367602348327637, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.6436151521540516, + "rescued": 2562, + "harmed": 1380, + "missed_rescue": 27, + "avoided_harm": 61, + "selected_both_correct": 4951, + "selected_both_wrong": 3628 + }, + { + "budget": 0.89, + "threshold": -3.3721210956573486, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.644036826200014, + "rescued": 2570, + "harmed": 1382, + "missed_rescue": 19, + "avoided_harm": 59, + "selected_both_correct": 5076, + "selected_both_wrong": 3635 + }, + { + "budget": 0.9, + "threshold": -3.5776143074035645, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.6438962681846933, + "rescued": 2575, + "harmed": 1389, + "missed_rescue": 14, + "avoided_harm": 52, + "selected_both_correct": 5197, + "selected_both_wrong": 3645 + }, + { + "budget": 0.91, + "threshold": -3.8324544429779053, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.6434043151310703, + "rescued": 2577, + "harmed": 1398, + "missed_rescue": 12, + "avoided_harm": 43, + "selected_both_correct": 5324, + "selected_both_wrong": 3649 + }, + { + "budget": 0.92, + "threshold": -4.133152008056641, + "operator": "ge", + "actual_fraction": 0.9199522102747909, + "metric": 0.6434043151310703, + "rescued": 2582, + "harmed": 1403, + "missed_rescue": 7, + "avoided_harm": 38, + "selected_both_correct": 5454, + "selected_both_wrong": 3651 + }, + { + "budget": 0.93, + "threshold": -4.427990913391113, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.6431934781080891, + "rescued": 2585, + "harmed": 1409, + "missed_rescue": 4, + "avoided_harm": 32, + "selected_both_correct": 5582, + "selected_both_wrong": 3656 + }, + { + "budget": 0.94, + "threshold": -4.858290195465088, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.6432637571157496, + "rescued": 2588, + "harmed": 1411, + "missed_rescue": 1, + "avoided_harm": 30, + "selected_both_correct": 5719, + "selected_both_wrong": 3657 + }, + { + "budget": 0.95, + "threshold": -5.234918594360352, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.6431231991004287, + "rescued": 2589, + "harmed": 1414, + "missed_rescue": 0, + "avoided_harm": 27, + "selected_both_correct": 5853, + "selected_both_wrong": 3661 + }, + { + "budget": 0.96, + "threshold": -5.675579071044922, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.6429123620774475, + "rescued": 2589, + "harmed": 1417, + "missed_rescue": 0, + "avoided_harm": 24, + "selected_both_correct": 5991, + "selected_both_wrong": 3662 + }, + { + "budget": 0.97, + "threshold": -6.235464572906494, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.6427015250544662, + "rescued": 2589, + "harmed": 1420, + "missed_rescue": 0, + "avoided_harm": 21, + "selected_both_correct": 6131, + "selected_both_wrong": 3662 + }, + { + "budget": 0.98, + "threshold": -6.9765944480896, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.6422095720008434, + "rescued": 2589, + "harmed": 1427, + "missed_rescue": 0, + "avoided_harm": 14, + "selected_both_correct": 6264, + "selected_both_wrong": 3664 + }, + { + "budget": 0.99, + "threshold": -8.352760314941406, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.6419284559702017, + "rescued": 2589, + "harmed": 1431, + "missed_rescue": 0, + "avoided_harm": 10, + "selected_both_correct": 6402, + "selected_both_wrong": 3664 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.6412256658935975, + "rescued": 2589, + "harmed": 1441, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 6535, + "selected_both_wrong": 3664 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.26688029946402747, + "rescued": 0, + "harmed": 0, + "missed_rescue": 10723, + "avoided_harm": 2500, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 15.251158714294434, + "operator": "ge", + "actual_fraction": 0.00703286731134617, + "metric": 0.2721833082834699, + "rescued": 187, + "harmed": 0, + "missed_rescue": 10536, + "avoided_harm": 2500, + "selected_both_correct": 0, + "selected_both_wrong": 61 + }, + { + "budget": 0.02, + "threshold": 14.266501426696777, + "operator": "ge", + "actual_fraction": 0.016476193176984375, + "metric": 0.2788475172276891, + "rescued": 423, + "harmed": 1, + "missed_rescue": 10300, + "avoided_harm": 2499, + "selected_both_correct": 0, + "selected_both_wrong": 157 + }, + { + "budget": 0.03, + "threshold": 13.713798522949219, + "operator": "ge", + "actual_fraction": 0.02489861894904007, + "metric": 0.284065451039333, + "rescued": 607, + "harmed": 1, + "missed_rescue": 10116, + "avoided_harm": 2499, + "selected_both_correct": 0, + "selected_both_wrong": 270 + }, + { + "budget": 0.04, + "threshold": 13.269920349121094, + "operator": "ge", + "actual_fraction": 0.03360462808042424, + "metric": 0.28970875988996964, + "rescued": 807, + "harmed": 2, + "missed_rescue": 9916, + "avoided_harm": 2498, + "selected_both_correct": 0, + "selected_both_wrong": 376 + }, + { + "budget": 0.05, + "threshold": 12.880468368530273, + "operator": "ge", + "actual_fraction": 0.043076312281995294, + "metric": 0.2952953520687406, + "rescued": 1006, + "harmed": 4, + "missed_rescue": 9717, + "avoided_harm": 2496, + "selected_both_correct": 1, + "selected_both_wrong": 508 + }, + { + "budget": 0.06, + "threshold": 12.571352005004883, + "operator": "ge", + "actual_fraction": 0.05178232141337946, + "metric": 0.3000028358335933, + "rescued": 1173, + "harmed": 5, + "missed_rescue": 9550, + "avoided_harm": 2495, + "selected_both_correct": 2, + "selected_both_wrong": 646 + }, + { + "budget": 0.07, + "threshold": 12.294123649597168, + "operator": "ge", + "actual_fraction": 0.060006238833905226, + "metric": 0.3049655446218416, + "rescued": 1349, + "harmed": 6, + "missed_rescue": 9374, + "avoided_harm": 2494, + "selected_both_correct": 2, + "selected_both_wrong": 759 + }, + { + "budget": 0.08, + "threshold": 12.034196853637695, + "operator": "ge", + "actual_fraction": 0.06786149788730397, + "metric": 0.30947452003516435, + "rescued": 1508, + "harmed": 6, + "missed_rescue": 9215, + "avoided_harm": 2494, + "selected_both_correct": 2, + "selected_both_wrong": 877 + }, + { + "budget": 0.09, + "threshold": 11.747140884399414, + "operator": "ge", + "actual_fraction": 0.07733318208887502, + "metric": 0.31443722882341263, + "rescued": 1685, + "harmed": 8, + "missed_rescue": 9038, + "avoided_harm": 2492, + "selected_both_correct": 2, + "selected_both_wrong": 1032 + }, + { + "budget": 0.1, + "threshold": 11.491235733032227, + "operator": "ge", + "actual_fraction": 0.08635113291552052, + "metric": 0.31917307092419817, + "rescued": 1853, + "harmed": 9, + "missed_rescue": 8870, + "avoided_harm": 2491, + "selected_both_correct": 2, + "selected_both_wrong": 1181 + }, + { + "budget": 0.11, + "threshold": 11.249296188354492, + "operator": "ge", + "actual_fraction": 0.09508550038283753, + "metric": 0.3233417463063267, + "rescued": 2004, + "harmed": 13, + "missed_rescue": 8719, + "avoided_harm": 2487, + "selected_both_correct": 2, + "selected_both_wrong": 1334 + }, + { + "budget": 0.12, + "threshold": 11.011529922485352, + "operator": "ge", + "actual_fraction": 0.10452882624847575, + "metric": 0.3284746051101721, + "rescued": 2186, + "harmed": 14, + "missed_rescue": 8537, + "avoided_harm": 2486, + "selected_both_correct": 6, + "selected_both_wrong": 1480 + }, + { + "budget": 0.13, + "threshold": 10.77489185333252, + "operator": "ge", + "actual_fraction": 0.11434081048124096, + "metric": 0.33349403057028615, + "rescued": 2365, + "harmed": 16, + "missed_rescue": 8358, + "avoided_harm": 2484, + "selected_both_correct": 8, + "selected_both_wrong": 1643 + }, + { + "budget": 0.14, + "threshold": 10.548977851867676, + "operator": "ge", + "actual_fraction": 0.12321696962822222, + "metric": 0.33769106428834755, + "rescued": 2518, + "harmed": 21, + "missed_rescue": 8205, + "avoided_harm": 2479, + "selected_both_correct": 13, + "selected_both_wrong": 1793 + }, + { + "budget": 0.15, + "threshold": 10.36727523803711, + "operator": "gt", + "actual_fraction": 0.13112894535348665, + "metric": 0.34109406460028924, + "rescued": 2640, + "harmed": 23, + "missed_rescue": 8083, + "avoided_harm": 2477, + "selected_both_correct": 15, + "selected_both_wrong": 1946 + }, + { + "budget": 0.16, + "threshold": 10.15395736694336, + "operator": "ge", + "actual_fraction": 0.14023197118793068, + "metric": 0.34560304001361203, + "rescued": 2808, + "harmed": 32, + "missed_rescue": 7915, + "avoided_harm": 2468, + "selected_both_correct": 18, + "selected_both_wrong": 2087 + }, + { + "budget": 0.17, + "threshold": 9.928215026855469, + "operator": "ge", + "actual_fraction": 0.1512350055298755, + "metric": 0.3507642571533902, + "rescued": 2994, + "harmed": 36, + "missed_rescue": 7729, + "avoided_harm": 2464, + "selected_both_correct": 19, + "selected_both_wrong": 2284 + }, + { + "budget": 0.18, + "threshold": 9.721980094909668, + "operator": "ge", + "actual_fraction": 0.1614440064657006, + "metric": 0.3552165158948473, + "rescued": 3156, + "harmed": 41, + "missed_rescue": 7567, + "avoided_harm": 2459, + "selected_both_correct": 22, + "selected_both_wrong": 2474 + }, + { + "budget": 0.19, + "threshold": 9.500179290771484, + "operator": "gt", + "actual_fraction": 0.1726739074951082, + "metric": 0.3603210163627598, + "rescued": 3343, + "harmed": 48, + "missed_rescue": 7380, + "avoided_harm": 2452, + "selected_both_correct": 27, + "selected_both_wrong": 2671 + }, + { + "budget": 0.2, + "threshold": 9.266512870788574, + "operator": "ge", + "actual_fraction": 0.1851799336414939, + "metric": 0.365680741854068, + "rescued": 3538, + "harmed": 54, + "missed_rescue": 7185, + "avoided_harm": 2446, + "selected_both_correct": 29, + "selected_both_wrong": 2909 + }, + { + "budget": 0.21, + "threshold": 9.037958145141602, + "operator": "gt", + "actual_fraction": 0.1977710347956782, + "metric": 0.3705016589626521, + "rescued": 3720, + "harmed": 66, + "missed_rescue": 7003, + "avoided_harm": 2434, + "selected_both_correct": 35, + "selected_both_wrong": 3153 + }, + { + "budget": 0.22, + "threshold": 8.772733688354492, + "operator": "ge", + "actual_fraction": 0.21416215296486402, + "metric": 0.3767972095397442, + "rescued": 3951, + "harmed": 75, + "missed_rescue": 6772, + "avoided_harm": 2425, + "selected_both_correct": 49, + "selected_both_wrong": 3477 + }, + { + "budget": 0.23, + "threshold": 8.523207664489746, + "operator": "ge", + "actual_fraction": 0.23041147945438561, + "metric": 0.3838300768510904, + "rescued": 4213, + "harmed": 89, + "missed_rescue": 6510, + "avoided_harm": 2411, + "selected_both_correct": 58, + "selected_both_wrong": 3765 + }, + { + "budget": 0.24, + "threshold": 8.271183967590332, + "operator": "ge", + "actual_fraction": 0.24697274763916854, + "metric": 0.3909480191702351, + "rescued": 4483, + "harmed": 108, + "missed_rescue": 6240, + "avoided_harm": 2392, + "selected_both_correct": 74, + "selected_both_wrong": 4044 + }, + { + "budget": 0.25, + "threshold": 7.994204521179199, + "operator": "ge", + "actual_fraction": 0.2668519411280946, + "metric": 0.39917193659076083, + "rescued": 4790, + "harmed": 125, + "missed_rescue": 5933, + "avoided_harm": 2375, + "selected_both_correct": 90, + "selected_both_wrong": 4405 + }, + { + "budget": 0.26, + "threshold": 7.755735397338867, + "operator": "ge", + "actual_fraction": 0.2847176927657885, + "metric": 0.4050704704647931, + "rescued": 5031, + "harmed": 158, + "missed_rescue": 5692, + "avoided_harm": 2342, + "selected_both_correct": 103, + "selected_both_wrong": 4748 + }, + { + "budget": 0.27, + "threshold": 7.53363561630249, + "operator": "ge", + "actual_fraction": 0.30207299435669116, + "metric": 0.41145109604968383, + "rescued": 5285, + "harmed": 187, + "missed_rescue": 5438, + "avoided_harm": 2313, + "selected_both_correct": 122, + "selected_both_wrong": 5058 + }, + { + "budget": 0.28, + "threshold": 7.3391337394714355, + "operator": "ge", + "actual_fraction": 0.31903127924453395, + "metric": 0.4179735133142387, + "rescued": 5539, + "harmed": 211, + "missed_rescue": 5184, + "avoided_harm": 2289, + "selected_both_correct": 139, + "selected_both_wrong": 5361 + }, + { + "budget": 0.29, + "threshold": 7.089168548583984, + "operator": "ge", + "actual_fraction": 0.3425686980687973, + "metric": 0.4269347474690185, + "rescued": 5905, + "harmed": 261, + "missed_rescue": 4818, + "avoided_harm": 2239, + "selected_both_correct": 171, + "selected_both_wrong": 5743 + }, + { + "budget": 0.3, + "threshold": 6.880478382110596, + "operator": "ge", + "actual_fraction": 0.3611717664407453, + "metric": 0.4334288063976406, + "rescued": 6169, + "harmed": 296, + "missed_rescue": 4554, + "avoided_harm": 2204, + "selected_both_correct": 193, + "selected_both_wrong": 6078 + }, + { + "budget": 0.31, + "threshold": 6.666043758392334, + "operator": "ge", + "actual_fraction": 0.3792076680940362, + "metric": 0.4392706235998072, + "rescued": 6416, + "harmed": 337, + "missed_rescue": 4307, + "avoided_harm": 2163, + "selected_both_correct": 220, + "selected_both_wrong": 6399 + }, + { + "budget": 0.32, + "threshold": 6.450697898864746, + "operator": "ge", + "actual_fraction": 0.3990868615829623, + "metric": 0.4447154240989139, + "rescued": 6659, + "harmed": 388, + "missed_rescue": 4064, + "avoided_harm": 2112, + "selected_both_correct": 255, + "selected_both_wrong": 6771 + }, + { + "budget": 0.33, + "threshold": 6.251960754394531, + "operator": "ge", + "actual_fraction": 0.4177750049627088, + "metric": 0.4512378413634688, + "rescued": 6926, + "harmed": 425, + "missed_rescue": 3797, + "avoided_harm": 2075, + "selected_both_correct": 288, + "selected_both_wrong": 7093 + }, + { + "budget": 0.34, + "threshold": 6.076109886169434, + "operator": "ge", + "actual_fraction": 0.43518702322547714, + "metric": 0.45622890848764996, + "rescued": 7140, + "harmed": 463, + "missed_rescue": 3583, + "avoided_harm": 2037, + "selected_both_correct": 329, + "selected_both_wrong": 7414 + }, + { + "budget": 0.35, + "threshold": 5.836214065551758, + "operator": "ge", + "actual_fraction": 0.4577319002920909, + "metric": 0.46340356747866035, + "rescued": 7441, + "harmed": 511, + "missed_rescue": 3282, + "avoided_harm": 1989, + "selected_both_correct": 369, + "selected_both_wrong": 7820 + }, + { + "budget": 0.36, + "threshold": 5.620694637298584, + "operator": "ge", + "actual_fraction": 0.4772707937498228, + "metric": 0.4683379179309758, + "rescued": 7666, + "harmed": 562, + "missed_rescue": 3057, + "avoided_harm": 1938, + "selected_both_correct": 420, + "selected_both_wrong": 8182 + }, + { + "budget": 0.37, + "threshold": 5.452563285827637, + "operator": "ge", + "actual_fraction": 0.4926410118254261, + "metric": 0.4718543515866489, + "rescued": 7832, + "harmed": 604, + "missed_rescue": 2891, + "avoided_harm": 1896, + "selected_both_correct": 458, + "selected_both_wrong": 8478 + }, + { + "budget": 0.38, + "threshold": 5.261322021484375, + "operator": "ge", + "actual_fraction": 0.5113858718770382, + "metric": 0.4762215353203074, + "rescued": 8053, + "harmed": 671, + "missed_rescue": 2670, + "avoided_harm": 1829, + "selected_both_correct": 516, + "selected_both_wrong": 8793 + }, + { + "budget": 0.39, + "threshold": 5.070874214172363, + "operator": "ge", + "actual_fraction": 0.530782973655106, + "metric": 0.48044692737430167, + "rescued": 8271, + "harmed": 740, + "missed_rescue": 2452, + "avoided_harm": 1760, + "selected_both_correct": 564, + "selected_both_wrong": 9142 + }, + { + "budget": 0.4, + "threshold": 4.91162109375, + "operator": "gt", + "actual_fraction": 0.5445367665825369, + "metric": 0.4838215693503105, + "rescued": 8432, + "harmed": 782, + "missed_rescue": 2291, + "avoided_harm": 1718, + "selected_both_correct": 604, + "selected_both_wrong": 9384 + }, + { + "budget": 0.41, + "threshold": 4.7608537673950195, + "operator": "ge", + "actual_fraction": 0.5588010095567592, + "metric": 0.48671411961546096, + "rescued": 8577, + "harmed": 825, + "missed_rescue": 2146, + "avoided_harm": 1675, + "selected_both_correct": 653, + "selected_both_wrong": 9650 + }, + { + "budget": 0.42, + "threshold": 4.58115816116333, + "operator": "ge", + "actual_fraction": 0.5735473442418398, + "metric": 0.48969174488840994, + "rescued": 8726, + "harmed": 869, + "missed_rescue": 1997, + "avoided_harm": 1631, + "selected_both_correct": 714, + "selected_both_wrong": 9916 + }, + { + "budget": 0.43, + "threshold": 4.429701805114746, + "operator": "ge", + "actual_fraction": 0.5872160621614724, + "metric": 0.4920171284349034, + "rescued": 8851, + "harmed": 912, + "missed_rescue": 1872, + "avoided_harm": 1588, + "selected_both_correct": 762, + "selected_both_wrong": 10182 + }, + { + "budget": 0.44, + "threshold": 4.268267631530762, + "operator": "ge", + "actual_fraction": 0.5994952216203954, + "metric": 0.4936619119190086, + "rescued": 8958, + "harmed": 961, + "missed_rescue": 1765, + "avoided_harm": 1539, + "selected_both_correct": 805, + "selected_both_wrong": 10416 + }, + { + "budget": 0.45, + "threshold": 4.120080471038818, + "operator": "ge", + "actual_fraction": 0.6116893060715197, + "metric": 0.495448487082778, + "rescued": 9070, + "harmed": 1010, + "missed_rescue": 1653, + "avoided_harm": 1490, + "selected_both_correct": 853, + "selected_both_wrong": 10637 + }, + { + "budget": 0.46, + "threshold": 3.951977014541626, + "operator": "ge", + "actual_fraction": 0.6253296656552193, + "metric": 0.49751864560587583, + "rescued": 9188, + "harmed": 1055, + "missed_rescue": 1535, + "avoided_harm": 1445, + "selected_both_correct": 909, + "selected_both_wrong": 10899 + }, + { + "budget": 0.47, + "threshold": 3.798562526702881, + "operator": "ge", + "actual_fraction": 0.6372118084110825, + "metric": 0.4993619374415109, + "rescued": 9292, + "harmed": 1094, + "missed_rescue": 1431, + "avoided_harm": 1406, + "selected_both_correct": 972, + "selected_both_wrong": 11112 + }, + { + "budget": 0.48, + "threshold": 3.635770797729492, + "operator": "ge", + "actual_fraction": 0.6485551427842214, + "metric": 0.5001559708476306, + "rescued": 9376, + "harmed": 1150, + "missed_rescue": 1347, + "avoided_harm": 1350, + "selected_both_correct": 1026, + "selected_both_wrong": 11318 + }, + { + "budget": 0.49, + "threshold": 3.44822359085083, + "operator": "ge", + "actual_fraction": 0.660919377250943, + "metric": 0.5014320959646088, + "rescued": 9463, + "harmed": 1192, + "missed_rescue": 1260, + "avoided_harm": 1308, + "selected_both_correct": 1090, + "selected_both_wrong": 11561 + }, + { + "budget": 0.5, + "threshold": 3.267157554626465, + "operator": "ge", + "actual_fraction": 0.6738791367722542, + "metric": 0.5030201627768482, + "rescued": 9569, + "harmed": 1242, + "missed_rescue": 1154, + "avoided_harm": 1258, + "selected_both_correct": 1164, + "selected_both_wrong": 11788 + }, + { + "budget": 0.51, + "threshold": 3.117006301879883, + "operator": "ge", + "actual_fraction": 0.6838612710206166, + "metric": 0.5040410628704307, + "rescued": 9643, + "harmed": 1280, + "missed_rescue": 1080, + "avoided_harm": 1220, + "selected_both_correct": 1233, + "selected_both_wrong": 11959 + }, + { + "budget": 0.52, + "threshold": 2.9639105796813965, + "operator": "ge", + "actual_fraction": 0.6931628052065905, + "metric": 0.5051470379718118, + "rescued": 9712, + "harmed": 1310, + "missed_rescue": 1011, + "avoided_harm": 1190, + "selected_both_correct": 1310, + "selected_both_wrong": 12111 + }, + { + "budget": 0.53, + "threshold": 2.8179378509521484, + "operator": "ge", + "actual_fraction": 0.7030882227830871, + "metric": 0.5060261463857301, + "rescued": 9777, + "harmed": 1344, + "missed_rescue": 946, + "avoided_harm": 1156, + "selected_both_correct": 1384, + "selected_both_wrong": 12288 + }, + { + "budget": 0.54, + "threshold": 2.663231372833252, + "operator": "ge", + "actual_fraction": 0.7130136403595837, + "metric": 0.5069052547996483, + "rescued": 9839, + "harmed": 1375, + "missed_rescue": 884, + "avoided_harm": 1125, + "selected_both_correct": 1476, + "selected_both_wrong": 12453 + }, + { + "budget": 0.55, + "threshold": 2.5109760761260986, + "operator": "ge", + "actual_fraction": 0.7221733828658934, + "metric": 0.5076425715339024, + "rescued": 9894, + "harmed": 1404, + "missed_rescue": 829, + "avoided_harm": 1096, + "selected_both_correct": 1540, + "selected_both_wrong": 12628 + }, + { + "budget": 0.56, + "threshold": 2.3562464714050293, + "operator": "ge", + "actual_fraction": 0.7308226753254119, + "metric": 0.5082380965884922, + "rescued": 9949, + "harmed": 1438, + "missed_rescue": 774, + "avoided_harm": 1062, + "selected_both_correct": 1617, + "selected_both_wrong": 12767 + }, + { + "budget": 0.57, + "threshold": 2.2040421962738037, + "operator": "ge", + "actual_fraction": 0.7396421178005275, + "metric": 0.5089754133227462, + "rescued": 10003, + "harmed": 1466, + "missed_rescue": 720, + "avoided_harm": 1034, + "selected_both_correct": 1700, + "selected_both_wrong": 12913 + }, + { + "budget": 0.58, + "threshold": 2.0592546463012695, + "operator": "ge", + "actual_fraction": 0.748092901908516, + "metric": 0.5096560133851346, + "rescued": 10051, + "harmed": 1490, + "missed_rescue": 672, + "avoided_harm": 1010, + "selected_both_correct": 1804, + "selected_both_wrong": 13035 + }, + { + "budget": 0.59, + "threshold": 1.9159198999404907, + "operator": "ge", + "actual_fraction": 0.7560332359697133, + "metric": 0.5100246717522616, + "rescued": 10094, + "harmed": 1520, + "missed_rescue": 629, + "avoided_harm": 980, + "selected_both_correct": 1882, + "selected_both_wrong": 13164 + }, + { + "budget": 0.6, + "threshold": 1.7584953308105469, + "operator": "ge", + "actual_fraction": 0.764455661741769, + "metric": 0.5102515384397244, + "rescued": 10139, + "harmed": 1557, + "missed_rescue": 584, + "avoided_harm": 943, + "selected_both_correct": 1978, + "selected_both_wrong": 13283 + }, + { + "budget": 0.61, + "threshold": 1.5953938961029053, + "operator": "ge", + "actual_fraction": 0.7731333125372203, + "metric": 0.5104216884553214, + "rescued": 10177, + "harmed": 1589, + "missed_rescue": 546, + "avoided_harm": 911, + "selected_both_correct": 2095, + "selected_both_wrong": 13402 + }, + { + "budget": 0.62, + "threshold": 1.4405584335327148, + "operator": "ge", + "actual_fraction": 0.7803646882000964, + "metric": 0.5107052718146499, + "rescued": 10211, + "harmed": 1613, + "missed_rescue": 512, + "avoided_harm": 887, + "selected_both_correct": 2187, + "selected_both_wrong": 13507 + }, + { + "budget": 0.63, + "threshold": 1.296458125114441, + "operator": "ge", + "actual_fraction": 0.7877945722145024, + "metric": 0.5108187051583813, + "rescued": 10241, + "harmed": 1639, + "missed_rescue": 482, + "avoided_harm": 861, + "selected_both_correct": 2307, + "selected_both_wrong": 13593 + }, + { + "budget": 0.64, + "threshold": 1.132359266281128, + "operator": "ge", + "actual_fraction": 0.7955931145960355, + "metric": 0.5112157218614412, + "rescued": 10273, + "harmed": 1657, + "missed_rescue": 450, + "avoided_harm": 843, + "selected_both_correct": 2419, + "selected_both_wrong": 13706 + }, + { + "budget": 0.65, + "threshold": 0.9609370827674866, + "operator": "ge", + "actual_fraction": 0.8029946402745087, + "metric": 0.5110172135099113, + "rescued": 10294, + "harmed": 1685, + "missed_rescue": 429, + "avoided_harm": 815, + "selected_both_correct": 2540, + "selected_both_wrong": 13797 + }, + { + "budget": 0.66, + "threshold": 0.797613799571991, + "operator": "ge", + "actual_fraction": 0.8107081076482432, + "metric": 0.5111306468536426, + "rescued": 10321, + "harmed": 1708, + "missed_rescue": 402, + "avoided_harm": 792, + "selected_both_correct": 2680, + "selected_both_wrong": 13879 + }, + { + "budget": 0.67, + "threshold": 0.6462197303771973, + "operator": "ge", + "actual_fraction": 0.8168051498738054, + "metric": 0.5113007968692397, + "rescued": 10342, + "harmed": 1723, + "missed_rescue": 381, + "avoided_harm": 777, + "selected_both_correct": 2781, + "selected_both_wrong": 13957 + }, + { + "budget": 0.68, + "threshold": 0.49411773681640625, + "operator": "ge", + "actual_fraction": 0.8232708504664946, + "metric": 0.5116694552363668, + "rescued": 10367, + "harmed": 1735, + "missed_rescue": 356, + "avoided_harm": 765, + "selected_both_correct": 2903, + "selected_both_wrong": 14026 + }, + { + "budget": 0.69, + "threshold": 0.3384990692138672, + "operator": "ge", + "actual_fraction": 0.8297649093951167, + "metric": 0.5118396052519638, + "rescued": 10391, + "harmed": 1753, + "missed_rescue": 332, + "avoided_harm": 747, + "selected_both_correct": 3022, + "selected_both_wrong": 14094 + }, + { + "budget": 0.7, + "threshold": 0.1719987392425537, + "operator": "ge", + "actual_fraction": 0.8369962850579928, + "metric": 0.5116978135722996, + "rescued": 10415, + "harmed": 1782, + "missed_rescue": 308, + "avoided_harm": 718, + "selected_both_correct": 3145, + "selected_both_wrong": 14173 + }, + { + "budget": 0.71, + "threshold": 0.0025517642498016357, + "operator": "ge", + "actual_fraction": 0.8437739273459434, + "metric": 0.5116127385645011, + "rescued": 10436, + "harmed": 1806, + "missed_rescue": 287, + "avoided_harm": 694, + "selected_both_correct": 3273, + "selected_both_wrong": 14239 + }, + { + "budget": 0.72, + "threshold": -0.1762939691543579, + "operator": "ge", + "actual_fraction": 0.8516575447352749, + "metric": 0.5117828885800981, + "rescued": 10464, + "harmed": 1828, + "missed_rescue": 259, + "avoided_harm": 672, + "selected_both_correct": 3437, + "selected_both_wrong": 14303 + }, + { + "budget": 0.73, + "threshold": -0.3592407703399658, + "operator": "ge", + "actual_fraction": 0.8595695204605394, + "metric": 0.511811246916031, + "rescued": 10487, + "harmed": 1850, + "missed_rescue": 236, + "avoided_harm": 650, + "selected_both_correct": 3590, + "selected_both_wrong": 14384 + }, + { + "budget": 0.74, + "threshold": -0.5128207206726074, + "operator": "ge", + "actual_fraction": 0.8667441794515498, + "metric": 0.511981396931628, + "rescued": 10511, + "harmed": 1868, + "missed_rescue": 212, + "avoided_harm": 632, + "selected_both_correct": 3738, + "selected_both_wrong": 14447 + }, + { + "budget": 0.75, + "threshold": -0.6840166449546814, + "operator": "ge", + "actual_fraction": 0.8740889884581573, + "metric": 0.5122649802909566, + "rescued": 10545, + "harmed": 1892, + "missed_rescue": 178, + "avoided_harm": 608, + "selected_both_correct": 3876, + "selected_both_wrong": 14510 + }, + { + "budget": 0.76, + "threshold": -0.8224172592163086, + "operator": "ge", + "actual_fraction": 0.8797606556447267, + "metric": 0.5123500552987551, + "rescued": 10561, + "harmed": 1905, + "missed_rescue": 162, + "avoided_harm": 595, + "selected_both_correct": 3990, + "selected_both_wrong": 14567 + }, + { + "budget": 0.77, + "threshold": -0.9985247850418091, + "operator": "ge", + "actual_fraction": 0.8864532229248787, + "metric": 0.5122649802909566, + "rescued": 10583, + "harmed": 1930, + "missed_rescue": 140, + "avoided_harm": 570, + "selected_both_correct": 4125, + "selected_both_wrong": 14621 + }, + { + "budget": 0.78, + "threshold": -1.168760895729065, + "operator": "ge", + "actual_fraction": 0.8934577319002921, + "metric": 0.511981396931628, + "rescued": 10600, + "harmed": 1957, + "missed_rescue": 123, + "avoided_harm": 543, + "selected_both_correct": 4272, + "selected_both_wrong": 14677 + }, + { + "budget": 0.79, + "threshold": -1.3431593179702759, + "operator": "ge", + "actual_fraction": 0.9004338825397725, + "metric": 0.5117545302441653, + "rescued": 10614, + "harmed": 1979, + "missed_rescue": 109, + "avoided_harm": 521, + "selected_both_correct": 4432, + "selected_both_wrong": 14727 + }, + { + "budget": 0.8, + "threshold": -1.5382206439971924, + "operator": "ge", + "actual_fraction": 0.907608541530783, + "metric": 0.5114425885489039, + "rescued": 10626, + "harmed": 2002, + "missed_rescue": 97, + "avoided_harm": 498, + "selected_both_correct": 4587, + "selected_both_wrong": 14790 + }, + { + "budget": 0.81, + "threshold": -1.7282600402832031, + "operator": "ge", + "actual_fraction": 0.9135921504126138, + "metric": 0.5109604968380456, + "rescued": 10639, + "harmed": 2032, + "missed_rescue": 84, + "avoided_harm": 468, + "selected_both_correct": 4710, + "selected_both_wrong": 14835 + }, + { + "budget": 0.82, + "threshold": -1.8991161584854126, + "operator": "ge", + "actual_fraction": 0.9198309843178403, + "metric": 0.510875421830247, + "rescued": 10655, + "harmed": 2051, + "missed_rescue": 68, + "avoided_harm": 449, + "selected_both_correct": 4858, + "selected_both_wrong": 14872 + }, + { + "budget": 0.83, + "threshold": -2.085026979446411, + "operator": "ge", + "actual_fraction": 0.9259280265434024, + "metric": 0.5104784051271871, + "rescued": 10666, + "harmed": 2076, + "missed_rescue": 57, + "avoided_harm": 424, + "selected_both_correct": 5002, + "selected_both_wrong": 14907 + }, + { + "budget": 0.84, + "threshold": -2.285501003265381, + "operator": "ge", + "actual_fraction": 0.9316847687377705, + "metric": 0.5099112384085301, + "rescued": 10673, + "harmed": 2103, + "missed_rescue": 50, + "avoided_harm": 397, + "selected_both_correct": 5144, + "selected_both_wrong": 14934 + }, + { + "budget": 0.85, + "threshold": -2.495553970336914, + "operator": "ge", + "actual_fraction": 0.9371295692368772, + "metric": 0.509570938377336, + "rescued": 10681, + "harmed": 2123, + "missed_rescue": 42, + "avoided_harm": 377, + "selected_both_correct": 5283, + "selected_both_wrong": 14959 + }, + { + "budget": 0.86, + "threshold": -2.6960723400115967, + "operator": "ge", + "actual_fraction": 0.9417236196579984, + "metric": 0.5090604883305447, + "rescued": 10683, + "harmed": 2143, + "missed_rescue": 40, + "avoided_harm": 357, + "selected_both_correct": 5398, + "selected_both_wrong": 14984 + }, + { + "budget": 0.87, + "threshold": -2.9289634227752686, + "operator": "ge", + "actual_fraction": 0.9468848367977767, + "metric": 0.5083515299322235, + "rescued": 10686, + "harmed": 2171, + "missed_rescue": 37, + "avoided_harm": 329, + "selected_both_correct": 5527, + "selected_both_wrong": 15006 + }, + { + "budget": 0.88, + "threshold": -3.1367602348327637, + "operator": "ge", + "actual_fraction": 0.951677395570428, + "metric": 0.5079545132291637, + "rescued": 10690, + "harmed": 2189, + "missed_rescue": 33, + "avoided_harm": 311, + "selected_both_correct": 5655, + "selected_both_wrong": 15025 + }, + { + "budget": 0.89, + "threshold": -3.3721210956573486, + "operator": "ge", + "actual_fraction": 0.9565266710149448, + "metric": 0.5075007798542381, + "rescued": 10696, + "harmed": 2211, + "missed_rescue": 27, + "avoided_harm": 289, + "selected_both_correct": 5786, + "selected_both_wrong": 15037 + }, + { + "budget": 0.9, + "threshold": -3.5776143074035645, + "operator": "ge", + "actual_fraction": 0.9605819130533421, + "metric": 0.5069336131355812, + "rescued": 10702, + "harmed": 2237, + "missed_rescue": 21, + "avoided_harm": 263, + "selected_both_correct": 5882, + "selected_both_wrong": 15052 + }, + { + "budget": 0.91, + "threshold": -3.8324544429779053, + "operator": "ge", + "actual_fraction": 0.9656296968493889, + "metric": 0.5063380880809915, + "rescued": 10708, + "harmed": 2264, + "missed_rescue": 15, + "avoided_harm": 236, + "selected_both_correct": 6012, + "selected_both_wrong": 15067 + }, + { + "budget": 0.92, + "threshold": -4.133152008056641, + "operator": "ge", + "actual_fraction": 0.9710177806766299, + "metric": 0.5054022629952074, + "rescued": 10712, + "harmed": 2301, + "missed_rescue": 11, + "avoided_harm": 199, + "selected_both_correct": 6147, + "selected_both_wrong": 15081 + }, + { + "budget": 0.93, + "threshold": -4.427990913391113, + "operator": "ge", + "actual_fraction": 0.975640189433684, + "metric": 0.5047500212687519, + "rescued": 10714, + "harmed": 2326, + "missed_rescue": 9, + "avoided_harm": 174, + "selected_both_correct": 6272, + "selected_both_wrong": 15092 + }, + { + "budget": 0.94, + "threshold": -4.858290195465088, + "operator": "ge", + "actual_fraction": 0.9819073816748434, + "metric": 0.5037574795111023, + "rescued": 10718, + "harmed": 2365, + "missed_rescue": 5, + "avoided_harm": 135, + "selected_both_correct": 6442, + "selected_both_wrong": 15100 + }, + { + "budget": 0.95, + "threshold": -5.234918594360352, + "operator": "ge", + "actual_fraction": 0.9857073986898449, + "metric": 0.5031052377846468, + "rescued": 10721, + "harmed": 2391, + "missed_rescue": 2, + "avoided_harm": 109, + "selected_both_correct": 6545, + "selected_both_wrong": 15102 + }, + { + "budget": 0.96, + "threshold": -5.675579071044922, + "operator": "ge", + "actual_fraction": 0.9896775657204435, + "metric": 0.502311204378527, + "rescued": 10723, + "harmed": 2421, + "missed_rescue": 0, + "avoided_harm": 79, + "selected_both_correct": 6641, + "selected_both_wrong": 15114 + }, + { + "budget": 0.97, + "threshold": -6.235464572906494, + "operator": "ge", + "actual_fraction": 0.9928820576808552, + "metric": 0.5017440376598701, + "rescued": 10723, + "harmed": 2441, + "missed_rescue": 0, + "avoided_harm": 59, + "selected_both_correct": 6729, + "selected_both_wrong": 15119 + }, + { + "budget": 0.98, + "threshold": -6.9765944480896, + "operator": "ge", + "actual_fraction": 0.9958880412897371, + "metric": 0.5012903042849446, + "rescued": 10723, + "harmed": 2457, + "missed_rescue": 0, + "avoided_harm": 43, + "selected_both_correct": 6813, + "selected_both_wrong": 15125 + }, + { + "budget": 0.99, + "threshold": -8.352760314941406, + "operator": "ge", + "actual_fraction": 0.9980716331565663, + "metric": 0.5009500042537504, + "rescued": 10723, + "harmed": 2469, + "missed_rescue": 0, + "avoided_harm": 31, + "selected_both_correct": 6876, + "selected_both_wrong": 15127 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5000708958398321, + "rescued": 10723, + "harmed": 2500, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 6911, + "selected_both_wrong": 15129 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 6535, + "rescuable": 2589, + "harmful": 1441, + "both_wrong": 3664, + "unit": "sample", + "total": 14229 + }, + "test": { + "both_correct": 6911, + "rescuable": 10723, + "harmful": 2500, + "both_wrong": 15129, + "unit": "sample", + "total": 35263 + } + }, + "small": { + "validation": 0.5605453650994447, + "test": 0.26688029946402747 + }, + "large": { + "validation": 0.6412256658935975, + "test": 0.5000708958398321 + }, + "learned_minus_confidence_test": -0.04060913705583752, + "learned_minus_small_test": 0.1893486090236225, + "learned_minus_large_test": -0.043841987352182143 + }, + { + "key": [ + "grefcoco", + "groundingdino", + "Qwen3.8" + ], + "expert_display": "GroundingDINO", + "run": "shared_12_groundingdino_Qwen3.8", + "validation_rows": 14229, + "test_rows": 35263, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detection_count", + "max_score", + "mean_score", + "query_count", + "detection_count_valid", + "max_score_valid", + "mean_score_valid", + "query_count_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/groundingdino/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 99, + "validation_area_0_25": 0.5648886077728582, + "validation_point": { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.7994237121371847, + "rescued": 4531, + "harmed": 1132, + "missed_rescue": 95, + "avoided_harm": 1, + "selected_both_correct": 6840, + "selected_both_wrong": 1583 + }, + "test_point": { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9873805405098829, + "metric": 0.7387062927147435, + "rescued": 18604, + "harmed": 1966, + "missed_rescue": 216, + "avoided_harm": 4, + "selected_both_correct": 7360, + "selected_both_wrong": 6888 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.5605453650994447, + "rescued": 0, + "harmed": 0, + "missed_rescue": 4626, + "avoided_harm": 1133, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.1584138125181198, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.5602642490688031, + "rescued": 0, + "harmed": 4, + "missed_rescue": 4626, + "avoided_harm": 1129, + "selected_both_correct": 138, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -0.21403978765010834, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.5601939700611427, + "rescued": 0, + "harmed": 5, + "missed_rescue": 4626, + "avoided_harm": 1128, + "selected_both_correct": 279, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -0.24253316223621368, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.5596317379998594, + "rescued": 0, + "harmed": 13, + "missed_rescue": 4626, + "avoided_harm": 1120, + "selected_both_correct": 410, + "selected_both_wrong": 3 + }, + { + "budget": 0.04, + "threshold": -0.2599165141582489, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.5594209009768782, + "rescued": 0, + "harmed": 16, + "missed_rescue": 4626, + "avoided_harm": 1117, + "selected_both_correct": 549, + "selected_both_wrong": 4 + }, + { + "budget": 0.05, + "threshold": -0.2725968658924103, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.5591397849462365, + "rescued": 3, + "harmed": 23, + "missed_rescue": 4623, + "avoided_harm": 1110, + "selected_both_correct": 675, + "selected_both_wrong": 10 + }, + { + "budget": 0.06, + "threshold": -0.2842792272567749, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.5588586689155949, + "rescued": 5, + "harmed": 29, + "missed_rescue": 4621, + "avoided_harm": 1104, + "selected_both_correct": 805, + "selected_both_wrong": 14 + }, + { + "budget": 0.07, + "threshold": -0.2935555577278137, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.5586478318926137, + "rescued": 12, + "harmed": 39, + "missed_rescue": 4614, + "avoided_harm": 1094, + "selected_both_correct": 926, + "selected_both_wrong": 19 + }, + { + "budget": 0.08, + "threshold": -0.30114513635635376, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.5589289479232553, + "rescued": 19, + "harmed": 42, + "missed_rescue": 4607, + "avoided_harm": 1091, + "selected_both_correct": 1053, + "selected_both_wrong": 24 + }, + { + "budget": 0.09, + "threshold": -0.308601438999176, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.5594209009768782, + "rescued": 34, + "harmed": 50, + "missed_rescue": 4592, + "avoided_harm": 1083, + "selected_both_correct": 1165, + "selected_both_wrong": 31 + }, + { + "budget": 0.1, + "threshold": -0.3146189749240875, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.559561458992199, + "rescued": 48, + "harmed": 62, + "missed_rescue": 4578, + "avoided_harm": 1071, + "selected_both_correct": 1275, + "selected_both_wrong": 37 + }, + { + "budget": 0.11, + "threshold": -0.32043084502220154, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.5599831330381615, + "rescued": 64, + "harmed": 72, + "missed_rescue": 4562, + "avoided_harm": 1061, + "selected_both_correct": 1382, + "selected_both_wrong": 47 + }, + { + "budget": 0.12, + "threshold": -0.32644838094711304, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.5604750860917844, + "rescued": 84, + "harmed": 85, + "missed_rescue": 4542, + "avoided_harm": 1048, + "selected_both_correct": 1485, + "selected_both_wrong": 53 + }, + { + "budget": 0.13, + "threshold": -0.33121857047080994, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.5616698292220114, + "rescued": 106, + "harmed": 90, + "missed_rescue": 4520, + "avoided_harm": 1043, + "selected_both_correct": 1592, + "selected_both_wrong": 61 + }, + { + "budget": 0.14, + "threshold": -0.33570271730422974, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.562794293344578, + "rescued": 126, + "harmed": 94, + "missed_rescue": 4500, + "avoided_harm": 1039, + "selected_both_correct": 1698, + "selected_both_wrong": 74 + }, + { + "budget": 0.15, + "threshold": -0.3407592475414276, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.5637781994518237, + "rescued": 151, + "harmed": 105, + "missed_rescue": 4475, + "avoided_harm": 1028, + "selected_both_correct": 1795, + "selected_both_wrong": 83 + }, + { + "budget": 0.16, + "threshold": -0.34456855058670044, + "operator": "ge", + "actual_fraction": 0.15995502143509735, + "metric": 0.5646918265514091, + "rescued": 180, + "harmed": 121, + "missed_rescue": 4446, + "avoided_harm": 1012, + "selected_both_correct": 1888, + "selected_both_wrong": 87 + }, + { + "budget": 0.17, + "threshold": -0.3487326204776764, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.5665190807505798, + "rescued": 211, + "harmed": 126, + "missed_rescue": 4415, + "avoided_harm": 1007, + "selected_both_correct": 1981, + "selected_both_wrong": 100 + }, + { + "budget": 0.18, + "threshold": -0.35280776023864746, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.5680652189191089, + "rescued": 242, + "harmed": 135, + "missed_rescue": 4384, + "avoided_harm": 998, + "selected_both_correct": 2070, + "selected_both_wrong": 114 + }, + { + "budget": 0.19, + "threshold": -0.3564819097518921, + "operator": "ge", + "actual_fraction": 0.18996415770609318, + "metric": 0.5697519151029588, + "rescued": 276, + "harmed": 145, + "missed_rescue": 4350, + "avoided_harm": 988, + "selected_both_correct": 2159, + "selected_both_wrong": 123 + }, + { + "budget": 0.2, + "threshold": -0.35992002487182617, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.5714386112868086, + "rescued": 309, + "harmed": 154, + "missed_rescue": 4317, + "avoided_harm": 979, + "selected_both_correct": 2246, + "selected_both_wrong": 136 + }, + { + "budget": 0.21, + "threshold": -0.36340460181236267, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.5737578185396022, + "rescued": 351, + "harmed": 163, + "missed_rescue": 4275, + "avoided_harm": 970, + "selected_both_correct": 2328, + "selected_both_wrong": 146 + }, + { + "budget": 0.22, + "threshold": -0.3674520254135132, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.5751633986928104, + "rescued": 384, + "harmed": 176, + "missed_rescue": 4242, + "avoided_harm": 957, + "selected_both_correct": 2410, + "selected_both_wrong": 160 + }, + { + "budget": 0.23, + "threshold": -0.37108901143074036, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.5764986998383583, + "rescued": 414, + "harmed": 187, + "missed_rescue": 4212, + "avoided_harm": 946, + "selected_both_correct": 2499, + "selected_both_wrong": 172 + }, + { + "budget": 0.24, + "threshold": -0.3745429813861847, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.578325954037529, + "rescued": 452, + "harmed": 199, + "missed_rescue": 4174, + "avoided_harm": 934, + "selected_both_correct": 2577, + "selected_both_wrong": 186 + }, + { + "budget": 0.25, + "threshold": -0.3773327171802521, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.5805046032750018, + "rescued": 490, + "harmed": 206, + "missed_rescue": 4136, + "avoided_harm": 927, + "selected_both_correct": 2664, + "selected_both_wrong": 197 + }, + { + "budget": 0.26, + "threshold": -0.3803459703922272, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.5829643685431162, + "rescued": 536, + "harmed": 217, + "missed_rescue": 4090, + "avoided_harm": 916, + "selected_both_correct": 2736, + "selected_both_wrong": 210 + }, + { + "budget": 0.27, + "threshold": -0.38361650705337524, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.584651064726966, + "rescued": 575, + "harmed": 232, + "missed_rescue": 4051, + "avoided_harm": 901, + "selected_both_correct": 2811, + "selected_both_wrong": 223 + }, + { + "budget": 0.28, + "threshold": -0.38652583956718445, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.5876730620563637, + "rescued": 626, + "harmed": 240, + "missed_rescue": 4000, + "avoided_harm": 893, + "selected_both_correct": 2882, + "selected_both_wrong": 236 + }, + { + "budget": 0.29, + "threshold": -0.38952407240867615, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.5904842223627802, + "rescued": 676, + "harmed": 250, + "missed_rescue": 3950, + "avoided_harm": 883, + "selected_both_correct": 2954, + "selected_both_wrong": 246 + }, + { + "budget": 0.3, + "threshold": -0.39262744784355164, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.5935764986998383, + "rescued": 726, + "harmed": 256, + "missed_rescue": 3900, + "avoided_harm": 877, + "selected_both_correct": 3029, + "selected_both_wrong": 257 + }, + { + "budget": 0.31, + "threshold": -0.3954668939113617, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.5956145899219903, + "rescued": 765, + "harmed": 266, + "missed_rescue": 3861, + "avoided_harm": 867, + "selected_both_correct": 3104, + "selected_both_wrong": 275 + }, + { + "budget": 0.32, + "threshold": -0.3986983895301819, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.5973715651135006, + "rescued": 804, + "harmed": 280, + "missed_rescue": 3822, + "avoided_harm": 853, + "selected_both_correct": 3178, + "selected_both_wrong": 291 + }, + { + "budget": 0.33, + "threshold": -0.4014112949371338, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.5994096563356526, + "rescued": 849, + "harmed": 296, + "missed_rescue": 3777, + "avoided_harm": 837, + "selected_both_correct": 3248, + "selected_both_wrong": 302 + }, + { + "budget": 0.34, + "threshold": -0.4042409658432007, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.6013774685501441, + "rescued": 890, + "harmed": 309, + "missed_rescue": 3736, + "avoided_harm": 824, + "selected_both_correct": 3323, + "selected_both_wrong": 315 + }, + { + "budget": 0.35, + "threshold": -0.40724706649780273, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.6035561177876169, + "rescued": 932, + "harmed": 320, + "missed_rescue": 3694, + "avoided_harm": 813, + "selected_both_correct": 3396, + "selected_both_wrong": 332 + }, + { + "budget": 0.36, + "threshold": -0.40968337655067444, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.6046805819101835, + "rescued": 965, + "harmed": 337, + "missed_rescue": 3661, + "avoided_harm": 796, + "selected_both_correct": 3475, + "selected_both_wrong": 345 + }, + { + "budget": 0.37, + "threshold": -0.4123799502849579, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.6079836952702228, + "rescued": 1019, + "harmed": 344, + "missed_rescue": 3607, + "avoided_harm": 789, + "selected_both_correct": 3546, + "selected_both_wrong": 355 + }, + { + "budget": 0.38, + "threshold": -0.41514647006988525, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.611778761683885, + "rescued": 1079, + "harmed": 350, + "missed_rescue": 3547, + "avoided_harm": 783, + "selected_both_correct": 3611, + "selected_both_wrong": 367 + }, + { + "budget": 0.39, + "threshold": -0.4179389476776123, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.6131843418370932, + "rescued": 1121, + "harmed": 372, + "missed_rescue": 3505, + "avoided_harm": 761, + "selected_both_correct": 3675, + "selected_both_wrong": 381 + }, + { + "budget": 0.4, + "threshold": -0.42099592089653015, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.6156441071052077, + "rescued": 1166, + "harmed": 382, + "missed_rescue": 3460, + "avoided_harm": 751, + "selected_both_correct": 3747, + "selected_both_wrong": 396 + }, + { + "budget": 0.41, + "threshold": -0.4243226647377014, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.617401082296718, + "rescued": 1208, + "harmed": 399, + "missed_rescue": 3418, + "avoided_harm": 734, + "selected_both_correct": 3815, + "selected_both_wrong": 411 + }, + { + "budget": 0.42, + "threshold": -0.4275209903717041, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.6203528006184552, + "rescued": 1261, + "harmed": 410, + "missed_rescue": 3365, + "avoided_harm": 723, + "selected_both_correct": 3878, + "selected_both_wrong": 427 + }, + { + "budget": 0.43, + "threshold": -0.4307159185409546, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.6228828448942301, + "rescued": 1310, + "harmed": 423, + "missed_rescue": 3316, + "avoided_harm": 710, + "selected_both_correct": 3949, + "selected_both_wrong": 436 + }, + { + "budget": 0.44, + "threshold": -0.4336964190006256, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.625764284208307, + "rescued": 1364, + "harmed": 436, + "missed_rescue": 3262, + "avoided_harm": 697, + "selected_both_correct": 4015, + "selected_both_wrong": 445 + }, + { + "budget": 0.45, + "threshold": -0.4373021125793457, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.627661817415138, + "rescued": 1410, + "harmed": 455, + "missed_rescue": 3216, + "avoided_harm": 678, + "selected_both_correct": 4077, + "selected_both_wrong": 461 + }, + { + "budget": 0.46, + "threshold": -0.4404309391975403, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.6299107456602713, + "rescued": 1459, + "harmed": 472, + "missed_rescue": 3167, + "avoided_harm": 661, + "selected_both_correct": 4140, + "selected_both_wrong": 474 + }, + { + "budget": 0.47, + "threshold": -0.44344601035118103, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.6319488368824232, + "rescued": 1501, + "harmed": 485, + "missed_rescue": 3125, + "avoided_harm": 648, + "selected_both_correct": 4210, + "selected_both_wrong": 491 + }, + { + "budget": 0.48, + "threshold": -0.4465356171131134, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.6338463700892544, + "rescued": 1545, + "harmed": 502, + "missed_rescue": 3081, + "avoided_harm": 631, + "selected_both_correct": 4281, + "selected_both_wrong": 501 + }, + { + "budget": 0.49, + "threshold": -0.44965147972106934, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.6358844613114063, + "rescued": 1592, + "harmed": 520, + "missed_rescue": 3034, + "avoided_harm": 613, + "selected_both_correct": 4344, + "selected_both_wrong": 516 + }, + { + "budget": 0.5, + "threshold": -0.453143447637558, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.6390470166561248, + "rescued": 1648, + "harmed": 531, + "missed_rescue": 2978, + "avoided_harm": 602, + "selected_both_correct": 4405, + "selected_both_wrong": 530 + }, + { + "budget": 0.51, + "threshold": -0.4565357565879822, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.6419987349778621, + "rescued": 1702, + "harmed": 543, + "missed_rescue": 2924, + "avoided_harm": 590, + "selected_both_correct": 4469, + "selected_both_wrong": 542 + }, + { + "budget": 0.52, + "threshold": -0.45961302518844604, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.6445990582612974, + "rescued": 1747, + "harmed": 551, + "missed_rescue": 2879, + "avoided_harm": 582, + "selected_both_correct": 4543, + "selected_both_wrong": 558 + }, + { + "budget": 0.53, + "threshold": -0.46368208527565, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.6468479865064305, + "rescued": 1795, + "harmed": 567, + "missed_rescue": 2831, + "avoided_harm": 566, + "selected_both_correct": 4608, + "selected_both_wrong": 571 + }, + { + "budget": 0.54, + "threshold": -0.46793216466903687, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.6500808208588095, + "rescued": 1850, + "harmed": 576, + "missed_rescue": 2776, + "avoided_harm": 557, + "selected_both_correct": 4672, + "selected_both_wrong": 585 + }, + { + "budget": 0.55, + "threshold": -0.4718930721282959, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.653243376203528, + "rescued": 1906, + "harmed": 587, + "missed_rescue": 2720, + "avoided_harm": 546, + "selected_both_correct": 4731, + "selected_both_wrong": 601 + }, + { + "budget": 0.56, + "threshold": -0.47629430890083313, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.654438119333755, + "rescued": 1942, + "harmed": 606, + "missed_rescue": 2684, + "avoided_harm": 527, + "selected_both_correct": 4800, + "selected_both_wrong": 620 + }, + { + "budget": 0.57, + "threshold": -0.4799485504627228, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.6574601166631527, + "rescued": 1998, + "harmed": 619, + "missed_rescue": 2628, + "avoided_harm": 514, + "selected_both_correct": 4861, + "selected_both_wrong": 632 + }, + { + "budget": 0.58, + "threshold": -0.4838995635509491, + "operator": "ge", + "actual_fraction": 0.5799423712137185, + "metric": 0.6599901609389276, + "rescued": 2048, + "harmed": 633, + "missed_rescue": 2578, + "avoided_harm": 500, + "selected_both_correct": 4921, + "selected_both_wrong": 650 + }, + { + "budget": 0.59, + "threshold": -0.4886912405490875, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.6618876941457587, + "rescued": 2098, + "harmed": 656, + "missed_rescue": 2528, + "avoided_harm": 477, + "selected_both_correct": 4979, + "selected_both_wrong": 662 + }, + { + "budget": 0.6, + "threshold": -0.49332454800605774, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.6642771804062126, + "rescued": 2146, + "harmed": 670, + "missed_rescue": 2480, + "avoided_harm": 463, + "selected_both_correct": 5049, + "selected_both_wrong": 672 + }, + { + "budget": 0.61, + "threshold": -0.49750789999961853, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.6670883407126291, + "rescued": 2199, + "harmed": 683, + "missed_rescue": 2427, + "avoided_harm": 450, + "selected_both_correct": 5115, + "selected_both_wrong": 682 + }, + { + "budget": 0.62, + "threshold": -0.5024909377098083, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.668775036896479, + "rescued": 2238, + "harmed": 698, + "missed_rescue": 2388, + "avoided_harm": 435, + "selected_both_correct": 5185, + "selected_both_wrong": 700 + }, + { + "budget": 0.63, + "threshold": -0.5067122578620911, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.6707428491109706, + "rescued": 2280, + "harmed": 712, + "missed_rescue": 2346, + "avoided_harm": 421, + "selected_both_correct": 5258, + "selected_both_wrong": 714 + }, + { + "budget": 0.64, + "threshold": -0.512323796749115, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.6732728933867453, + "rescued": 2329, + "harmed": 725, + "missed_rescue": 2297, + "avoided_harm": 408, + "selected_both_correct": 5325, + "selected_both_wrong": 727 + }, + { + "budget": 0.65, + "threshold": -0.5170255303382874, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.675592100639539, + "rescued": 2377, + "harmed": 740, + "missed_rescue": 2249, + "avoided_harm": 393, + "selected_both_correct": 5386, + "selected_both_wrong": 745 + }, + { + "budget": 0.66, + "threshold": -0.5215396881103516, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.6772787968233889, + "rescued": 2417, + "harmed": 756, + "missed_rescue": 2209, + "avoided_harm": 377, + "selected_both_correct": 5453, + "selected_both_wrong": 765 + }, + { + "budget": 0.67, + "threshold": -0.5264327526092529, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.6788952139995783, + "rescued": 2457, + "harmed": 773, + "missed_rescue": 2169, + "avoided_harm": 360, + "selected_both_correct": 5522, + "selected_both_wrong": 781 + }, + { + "budget": 0.68, + "threshold": -0.5312389135360718, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.6810738632370511, + "rescued": 2504, + "harmed": 789, + "missed_rescue": 2122, + "avoided_harm": 344, + "selected_both_correct": 5585, + "selected_both_wrong": 797 + }, + { + "budget": 0.69, + "threshold": -0.5368650555610657, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.6833930704898447, + "rescued": 2550, + "harmed": 802, + "missed_rescue": 2076, + "avoided_harm": 331, + "selected_both_correct": 5652, + "selected_both_wrong": 814 + }, + { + "budget": 0.7, + "threshold": -0.5431069135665894, + "operator": "ge", + "actual_fraction": 0.6999789162977019, + "metric": 0.6867664628575445, + "rescued": 2614, + "harmed": 818, + "missed_rescue": 2012, + "avoided_harm": 315, + "selected_both_correct": 5698, + "selected_both_wrong": 830 + }, + { + "budget": 0.71, + "threshold": -0.5487136244773865, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.6895073441563006, + "rescued": 2664, + "harmed": 829, + "missed_rescue": 1962, + "avoided_harm": 304, + "selected_both_correct": 5762, + "selected_both_wrong": 847 + }, + { + "budget": 0.72, + "threshold": -0.5549414753913879, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.6909129243095088, + "rescued": 2704, + "harmed": 849, + "missed_rescue": 1922, + "avoided_harm": 284, + "selected_both_correct": 5830, + "selected_both_wrong": 861 + }, + { + "budget": 0.73, + "threshold": -0.5604677200317383, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.6918265514090941, + "rescued": 2739, + "harmed": 871, + "missed_rescue": 1887, + "avoided_harm": 262, + "selected_both_correct": 5903, + "selected_both_wrong": 874 + }, + { + "budget": 0.74, + "threshold": -0.5666090250015259, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.6940754796542273, + "rescued": 2787, + "harmed": 887, + "missed_rescue": 1839, + "avoided_harm": 246, + "selected_both_correct": 5960, + "selected_both_wrong": 895 + }, + { + "budget": 0.75, + "threshold": -0.574927568435669, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.6962541288917, + "rescued": 2831, + "harmed": 900, + "missed_rescue": 1795, + "avoided_harm": 233, + "selected_both_correct": 6025, + "selected_both_wrong": 915 + }, + { + "budget": 0.76, + "threshold": -0.5812664031982422, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.6987841731674749, + "rescued": 2879, + "harmed": 912, + "missed_rescue": 1747, + "avoided_harm": 221, + "selected_both_correct": 6094, + "selected_both_wrong": 929 + }, + { + "budget": 0.77, + "threshold": -0.5883752107620239, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.7008925433972872, + "rescued": 2923, + "harmed": 926, + "missed_rescue": 1703, + "avoided_harm": 207, + "selected_both_correct": 6166, + "selected_both_wrong": 941 + }, + { + "budget": 0.78, + "threshold": -0.5972744226455688, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.7033523086654017, + "rescued": 2977, + "harmed": 945, + "missed_rescue": 1649, + "avoided_harm": 188, + "selected_both_correct": 6220, + "selected_both_wrong": 956 + }, + { + "budget": 0.79, + "threshold": -0.6058458089828491, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.7058120739335161, + "rescued": 3026, + "harmed": 959, + "missed_rescue": 1600, + "avoided_harm": 174, + "selected_both_correct": 6288, + "selected_both_wrong": 967 + }, + { + "budget": 0.8, + "threshold": -0.6140543818473816, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.7068662590484223, + "rescued": 3067, + "harmed": 985, + "missed_rescue": 1559, + "avoided_harm": 148, + "selected_both_correct": 6341, + "selected_both_wrong": 990 + }, + { + "budget": 0.81, + "threshold": -0.6230268478393555, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.7082718392016305, + "rescued": 3109, + "harmed": 1007, + "missed_rescue": 1517, + "avoided_harm": 126, + "selected_both_correct": 6402, + "selected_both_wrong": 1007 + }, + { + "budget": 0.82, + "threshold": -0.6330366134643555, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.7100990934008011, + "rescued": 3152, + "harmed": 1024, + "missed_rescue": 1474, + "avoided_harm": 109, + "selected_both_correct": 6466, + "selected_both_wrong": 1025 + }, + { + "budget": 0.83, + "threshold": -0.6429333686828613, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.711434394546349, + "rescued": 3192, + "harmed": 1045, + "missed_rescue": 1434, + "avoided_harm": 88, + "selected_both_correct": 6528, + "selected_both_wrong": 1045 + }, + { + "budget": 0.84, + "threshold": -0.6513019800186157, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.7131913697378593, + "rescued": 3238, + "harmed": 1066, + "missed_rescue": 1388, + "avoided_harm": 67, + "selected_both_correct": 6584, + "selected_both_wrong": 1064 + }, + { + "budget": 0.85, + "threshold": -0.6624053120613098, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.7150889029446904, + "rescued": 3282, + "harmed": 1083, + "missed_rescue": 1344, + "avoided_harm": 50, + "selected_both_correct": 6639, + "selected_both_wrong": 1090 + }, + { + "budget": 0.86, + "threshold": -0.6759263277053833, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.7167755991285403, + "rescued": 3326, + "harmed": 1103, + "missed_rescue": 1300, + "avoided_harm": 30, + "selected_both_correct": 6698, + "selected_both_wrong": 1109 + }, + { + "budget": 0.87, + "threshold": -0.6904275417327881, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.7191650853889943, + "rescued": 3371, + "harmed": 1114, + "missed_rescue": 1255, + "avoided_harm": 19, + "selected_both_correct": 6772, + "selected_both_wrong": 1122 + }, + { + "budget": 0.88, + "threshold": -0.7045450210571289, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.7228195937873357, + "rescued": 3437, + "harmed": 1128, + "missed_rescue": 1189, + "avoided_harm": 5, + "selected_both_correct": 6814, + "selected_both_wrong": 1142 + }, + { + "budget": 0.89, + "threshold": -0.7205488681793213, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.7301286105840186, + "rescued": 3541, + "harmed": 1128, + "missed_rescue": 1085, + "avoided_harm": 5, + "selected_both_correct": 6815, + "selected_both_wrong": 1179 + }, + { + "budget": 0.9, + "threshold": -0.7357243299484253, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.7374376273807014, + "rescued": 3645, + "harmed": 1128, + "missed_rescue": 981, + "avoided_harm": 5, + "selected_both_correct": 6815, + "selected_both_wrong": 1218 + }, + { + "budget": 0.91, + "threshold": -0.7513164281845093, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.7445358071544029, + "rescued": 3746, + "harmed": 1128, + "missed_rescue": 880, + "avoided_harm": 5, + "selected_both_correct": 6815, + "selected_both_wrong": 1259 + }, + { + "budget": 0.92, + "threshold": -0.7671180963516235, + "operator": "ge", + "actual_fraction": 0.9199522102747909, + "metric": 0.7521962189893878, + "rescued": 3855, + "harmed": 1128, + "missed_rescue": 771, + "avoided_harm": 5, + "selected_both_correct": 6817, + "selected_both_wrong": 1290 + }, + { + "budget": 0.93, + "threshold": -0.7833515405654907, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.758872724717127, + "rescued": 3951, + "harmed": 1129, + "missed_rescue": 675, + "avoided_harm": 4, + "selected_both_correct": 6818, + "selected_both_wrong": 1334 + }, + { + "budget": 0.94, + "threshold": -0.8013419508934021, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.7654789514372057, + "rescued": 4045, + "harmed": 1129, + "missed_rescue": 581, + "avoided_harm": 4, + "selected_both_correct": 6821, + "selected_both_wrong": 1380 + }, + { + "budget": 0.95, + "threshold": -0.8188987374305725, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.7727879682338885, + "rescued": 4149, + "harmed": 1129, + "missed_rescue": 477, + "avoided_harm": 4, + "selected_both_correct": 6822, + "selected_both_wrong": 1417 + }, + { + "budget": 0.96, + "threshold": -0.8395025134086609, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.7791130789233256, + "rescued": 4240, + "harmed": 1130, + "missed_rescue": 386, + "avoided_harm": 3, + "selected_both_correct": 6826, + "selected_both_wrong": 1463 + }, + { + "budget": 0.97, + "threshold": -0.8628664612770081, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.7857193056434043, + "rescued": 4335, + "harmed": 1131, + "missed_rescue": 291, + "avoided_harm": 2, + "selected_both_correct": 6829, + "selected_both_wrong": 1507 + }, + { + "budget": 0.98, + "threshold": -0.8857419490814209, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.7927472064094455, + "rescued": 4435, + "harmed": 1131, + "missed_rescue": 191, + "avoided_harm": 2, + "selected_both_correct": 6835, + "selected_both_wrong": 1543 + }, + { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.7994237121371847, + "rescued": 4531, + "harmed": 1132, + "missed_rescue": 95, + "avoided_harm": 1, + "selected_both_correct": 6840, + "selected_both_wrong": 1583 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8060299388572634, + "rescued": 4626, + "harmed": 1133, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 6843, + "selected_both_wrong": 1627 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.26688029946402747, + "rescued": 0, + "harmed": 0, + "missed_rescue": 18820, + "avoided_harm": 1970, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.1584138125181198, + "operator": "ge", + "actual_fraction": 0.029095652667101495, + "metric": 0.2670788078155574, + "rescued": 15, + "harmed": 8, + "missed_rescue": 18805, + "avoided_harm": 1962, + "selected_both_correct": 994, + "selected_both_wrong": 9 + }, + { + "budget": 0.02, + "threshold": -0.21403978765010834, + "operator": "ge", + "actual_fraction": 0.04622408757054136, + "metric": 0.26747582451861723, + "rescued": 38, + "harmed": 17, + "missed_rescue": 18782, + "avoided_harm": 1953, + "selected_both_correct": 1519, + "selected_both_wrong": 56 + }, + { + "budget": 0.03, + "threshold": -0.24253316223621368, + "operator": "ge", + "actual_fraction": 0.056149505147037974, + "metric": 0.26821314125287127, + "rescued": 68, + "harmed": 21, + "missed_rescue": 18752, + "avoided_harm": 1949, + "selected_both_correct": 1793, + "selected_both_wrong": 98 + }, + { + "budget": 0.04, + "threshold": -0.2599165141582489, + "operator": "ge", + "actual_fraction": 0.06372118084110824, + "metric": 0.26883702464339393, + "rescued": 98, + "harmed": 29, + "missed_rescue": 18722, + "avoided_harm": 1941, + "selected_both_correct": 1984, + "selected_both_wrong": 136 + }, + { + "budget": 0.05, + "threshold": -0.2725968658924103, + "operator": "ge", + "actual_fraction": 0.06882568130902078, + "metric": 0.2692907580183195, + "rescued": 125, + "harmed": 40, + "missed_rescue": 18695, + "avoided_harm": 1930, + "selected_both_correct": 2087, + "selected_both_wrong": 175 + }, + { + "budget": 0.06, + "threshold": -0.2842792272567749, + "operator": "ge", + "actual_fraction": 0.07475257351898591, + "metric": 0.2701415080963049, + "rescued": 161, + "harmed": 46, + "missed_rescue": 18659, + "avoided_harm": 1924, + "selected_both_correct": 2200, + "selected_both_wrong": 229 + }, + { + "budget": 0.07, + "threshold": -0.2935555577278137, + "operator": "ge", + "actual_fraction": 0.08033916569775686, + "metric": 0.27119076652582025, + "rescued": 209, + "harmed": 57, + "missed_rescue": 18611, + "avoided_harm": 1913, + "selected_both_correct": 2298, + "selected_both_wrong": 269 + }, + { + "budget": 0.08, + "threshold": -0.30114513635635376, + "operator": "ge", + "actual_fraction": 0.08632277457958767, + "metric": 0.27260868332246263, + "rescued": 270, + "harmed": 68, + "missed_rescue": 18550, + "avoided_harm": 1902, + "selected_both_correct": 2381, + "selected_both_wrong": 325 + }, + { + "budget": 0.09, + "threshold": -0.308601438999176, + "operator": "ge", + "actual_fraction": 0.09227802512548564, + "metric": 0.27419675013470207, + "rescued": 338, + "harmed": 80, + "missed_rescue": 18482, + "avoided_harm": 1890, + "selected_both_correct": 2473, + "selected_both_wrong": 363 + }, + { + "budget": 0.1, + "threshold": -0.3146189749240875, + "operator": "ge", + "actual_fraction": 0.09783625896832374, + "metric": 0.275699741939143, + "rescued": 405, + "harmed": 94, + "missed_rescue": 18415, + "avoided_harm": 1876, + "selected_both_correct": 2542, + "selected_both_wrong": 409 + }, + { + "budget": 0.11, + "threshold": -0.32043084502220154, + "operator": "ge", + "actual_fraction": 0.10362135949862462, + "metric": 0.2776564671185095, + "rescued": 482, + "harmed": 102, + "missed_rescue": 18338, + "avoided_harm": 1868, + "selected_both_correct": 2610, + "selected_both_wrong": 460 + }, + { + "budget": 0.12, + "threshold": -0.32644838094711304, + "operator": "ge", + "actual_fraction": 0.11031392677877662, + "metric": 0.28032215069619715, + "rescued": 589, + "harmed": 115, + "missed_rescue": 18231, + "avoided_harm": 1855, + "selected_both_correct": 2673, + "selected_both_wrong": 513 + }, + { + "budget": 0.13, + "threshold": -0.33121857047080994, + "operator": "ge", + "actual_fraction": 0.11658111901993591, + "metric": 0.28267589257862347, + "rescued": 680, + "harmed": 123, + "missed_rescue": 18140, + "avoided_harm": 1847, + "selected_both_correct": 2746, + "selected_both_wrong": 562 + }, + { + "budget": 0.14, + "threshold": -0.33570271730422974, + "operator": "ge", + "actual_fraction": 0.12262144457363242, + "metric": 0.2851997844766469, + "rescued": 777, + "harmed": 131, + "missed_rescue": 18043, + "avoided_harm": 1839, + "selected_both_correct": 2805, + "selected_both_wrong": 611 + }, + { + "budget": 0.15, + "threshold": -0.3407592475414276, + "operator": "ge", + "actual_fraction": 0.13024983693956838, + "metric": 0.2880072597339988, + "rescued": 897, + "harmed": 152, + "missed_rescue": 17923, + "avoided_harm": 1818, + "selected_both_correct": 2870, + "selected_both_wrong": 674 + }, + { + "budget": 0.16, + "threshold": -0.34456855058670044, + "operator": "ge", + "actual_fraction": 0.13714091257125033, + "metric": 0.29106996001474633, + "rescued": 1018, + "harmed": 165, + "missed_rescue": 17802, + "avoided_harm": 1805, + "selected_both_correct": 2930, + "selected_both_wrong": 723 + }, + { + "budget": 0.17, + "threshold": -0.3487326204776764, + "operator": "ge", + "actual_fraction": 0.143521538156141, + "metric": 0.2940475852876953, + "rescued": 1137, + "harmed": 179, + "missed_rescue": 17683, + "avoided_harm": 1791, + "selected_both_correct": 2975, + "selected_both_wrong": 770 + }, + { + "budget": 0.18, + "threshold": -0.35280776023864746, + "operator": "ge", + "actual_fraction": 0.15095142217054702, + "metric": 0.29747894393556984, + "rescued": 1269, + "harmed": 190, + "missed_rescue": 17551, + "avoided_harm": 1780, + "selected_both_correct": 3022, + "selected_both_wrong": 842 + }, + { + "budget": 0.19, + "threshold": -0.3564819097518921, + "operator": "ge", + "actual_fraction": 0.15823951450528884, + "metric": 0.3003714942007203, + "rescued": 1395, + "harmed": 214, + "missed_rescue": 17425, + "avoided_harm": 1756, + "selected_both_correct": 3075, + "selected_both_wrong": 896 + }, + { + "budget": 0.2, + "threshold": -0.35992002487182617, + "operator": "ge", + "actual_fraction": 0.16496044012137367, + "metric": 0.30334911947366927, + "rescued": 1511, + "harmed": 225, + "missed_rescue": 17309, + "avoided_harm": 1745, + "selected_both_correct": 3127, + "selected_both_wrong": 954 + }, + { + "budget": 0.21, + "threshold": -0.36340460181236267, + "operator": "ge", + "actual_fraction": 0.17290077418257097, + "metric": 0.30697898647307376, + "rescued": 1651, + "harmed": 237, + "missed_rescue": 17169, + "avoided_harm": 1733, + "selected_both_correct": 3185, + "selected_both_wrong": 1024 + }, + { + "budget": 0.22, + "threshold": -0.3674520254135132, + "operator": "ge", + "actual_fraction": 0.1829679834387318, + "metric": 0.3114879618863965, + "rescued": 1832, + "harmed": 259, + "missed_rescue": 16988, + "avoided_harm": 1711, + "selected_both_correct": 3252, + "selected_both_wrong": 1109 + }, + { + "budget": 0.23, + "threshold": -0.37108901143074036, + "operator": "ge", + "actual_fraction": 0.19122025919519042, + "metric": 0.31480588719053965, + "rescued": 1970, + "harmed": 280, + "missed_rescue": 16850, + "avoided_harm": 1690, + "selected_both_correct": 3314, + "selected_both_wrong": 1179 + }, + { + "budget": 0.24, + "threshold": -0.3745429813861847, + "operator": "ge", + "actual_fraction": 0.1999262683265746, + "metric": 0.3192581459319967, + "rescued": 2144, + "harmed": 297, + "missed_rescue": 16676, + "avoided_harm": 1673, + "selected_both_correct": 3372, + "selected_both_wrong": 1237 + }, + { + "budget": 0.25, + "threshold": -0.3773327171802521, + "operator": "ge", + "actual_fraction": 0.2080651107393018, + "metric": 0.32294472960326687, + "rescued": 2294, + "harmed": 317, + "missed_rescue": 16526, + "avoided_harm": 1653, + "selected_both_correct": 3423, + "selected_both_wrong": 1303 + }, + { + "budget": 0.26, + "threshold": -0.3803459703922272, + "operator": "ge", + "actual_fraction": 0.21603380313643195, + "metric": 0.3270283299775969, + "rescued": 2457, + "harmed": 336, + "missed_rescue": 16363, + "avoided_harm": 1634, + "selected_both_correct": 3465, + "selected_both_wrong": 1360 + }, + { + "budget": 0.27, + "threshold": -0.38361650705337524, + "operator": "ge", + "actual_fraction": 0.2243144372288234, + "metric": 0.33170745540651675, + "rescued": 2634, + "harmed": 348, + "missed_rescue": 16186, + "avoided_harm": 1622, + "selected_both_correct": 3522, + "selected_both_wrong": 1406 + }, + { + "budget": 0.28, + "threshold": -0.38652583956718445, + "operator": "ge", + "actual_fraction": 0.2324249213056178, + "metric": 0.33542239741371976, + "rescued": 2793, + "harmed": 376, + "missed_rescue": 16027, + "avoided_harm": 1594, + "selected_both_correct": 3554, + "selected_both_wrong": 1473 + }, + { + "budget": 0.29, + "threshold": -0.38952407240867615, + "operator": "ge", + "actual_fraction": 0.2419816805149874, + "metric": 0.3401015228426396, + "rescued": 2975, + "harmed": 393, + "missed_rescue": 15845, + "avoided_harm": 1577, + "selected_both_correct": 3623, + "selected_both_wrong": 1542 + }, + { + "budget": 0.3, + "threshold": -0.39262744784355164, + "operator": "ge", + "actual_fraction": 0.2512832147009613, + "metric": 0.3448373649434251, + "rescued": 3169, + "harmed": 420, + "missed_rescue": 15651, + "avoided_harm": 1550, + "selected_both_correct": 3674, + "selected_both_wrong": 1598 + }, + { + "budget": 0.31, + "threshold": -0.3954668939113617, + "operator": "ge", + "actual_fraction": 0.2596772821370842, + "metric": 0.3491194736692851, + "rescued": 3340, + "harmed": 440, + "missed_rescue": 15480, + "avoided_harm": 1530, + "selected_both_correct": 3706, + "selected_both_wrong": 1671 + }, + { + "budget": 0.32, + "threshold": -0.3986983895301819, + "operator": "ge", + "actual_fraction": 0.26985792473697645, + "metric": 0.35459263250432466, + "rescued": 3553, + "harmed": 460, + "missed_rescue": 15267, + "avoided_harm": 1510, + "selected_both_correct": 3758, + "selected_both_wrong": 1745 + }, + { + "budget": 0.33, + "threshold": -0.4014112949371338, + "operator": "ge", + "actual_fraction": 0.27757139211071097, + "metric": 0.35878966622238606, + "rescued": 3718, + "harmed": 477, + "missed_rescue": 15102, + "avoided_harm": 1493, + "selected_both_correct": 3788, + "selected_both_wrong": 1805 + }, + { + "budget": 0.34, + "threshold": -0.4042409658432007, + "operator": "ge", + "actual_fraction": 0.28729830133567763, + "metric": 0.3635255083231716, + "rescued": 3913, + "harmed": 505, + "missed_rescue": 14907, + "avoided_harm": 1465, + "selected_both_correct": 3845, + "selected_both_wrong": 1868 + }, + { + "budget": 0.35, + "threshold": -0.40724706649780273, + "operator": "ge", + "actual_fraction": 0.29793267731049544, + "metric": 0.36928225051753966, + "rescued": 4137, + "harmed": 526, + "missed_rescue": 14683, + "avoided_harm": 1444, + "selected_both_correct": 3903, + "selected_both_wrong": 1940 + }, + { + "budget": 0.36, + "threshold": -0.40968337655067444, + "operator": "ge", + "actual_fraction": 0.3067237614496781, + "metric": 0.3743583926495193, + "rescued": 4332, + "harmed": 542, + "missed_rescue": 14488, + "avoided_harm": 1428, + "selected_both_correct": 3949, + "selected_both_wrong": 1993 + }, + { + "budget": 0.37, + "threshold": -0.4123799502849579, + "operator": "ge", + "actual_fraction": 0.3161387289793835, + "metric": 0.3794912514533647, + "rescued": 4536, + "harmed": 565, + "missed_rescue": 14284, + "avoided_harm": 1405, + "selected_both_correct": 3993, + "selected_both_wrong": 2054 + }, + { + "budget": 0.38, + "threshold": -0.41514647006988525, + "operator": "ge", + "actual_fraction": 0.32538354649349177, + "metric": 0.3843121685619488, + "rescued": 4725, + "harmed": 584, + "missed_rescue": 14095, + "avoided_harm": 1386, + "selected_both_correct": 4050, + "selected_both_wrong": 2115 + }, + { + "budget": 0.39, + "threshold": -0.4179389476776123, + "operator": "ge", + "actual_fraction": 0.3354507557496526, + "metric": 0.389671894053257, + "rescued": 4940, + "harmed": 610, + "missed_rescue": 13880, + "avoided_harm": 1360, + "selected_both_correct": 4104, + "selected_both_wrong": 2175 + }, + { + "budget": 0.4, + "threshold": -0.42099592089653015, + "operator": "ge", + "actual_fraction": 0.34653886509939597, + "metric": 0.395428636247625, + "rescued": 5169, + "harmed": 636, + "missed_rescue": 13651, + "avoided_harm": 1334, + "selected_both_correct": 4154, + "selected_both_wrong": 2261 + }, + { + "budget": 0.41, + "threshold": -0.4243226647377014, + "operator": "ge", + "actual_fraction": 0.3577404077928707, + "metric": 0.4010719450982616, + "rescued": 5399, + "harmed": 667, + "missed_rescue": 13421, + "avoided_harm": 1303, + "selected_both_correct": 4211, + "selected_both_wrong": 2338 + }, + { + "budget": 0.42, + "threshold": -0.4275209903717041, + "operator": "ge", + "actual_fraction": 0.3693673255253382, + "metric": 0.40776451237841366, + "rescued": 5657, + "harmed": 689, + "missed_rescue": 13163, + "avoided_harm": 1281, + "selected_both_correct": 4262, + "selected_both_wrong": 2417 + }, + { + "budget": 0.43, + "threshold": -0.4307159185409546, + "operator": "ge", + "actual_fraction": 0.38039871820321586, + "metric": 0.41423021297110285, + "rescued": 5901, + "harmed": 705, + "missed_rescue": 12919, + "avoided_harm": 1265, + "selected_both_correct": 4323, + "selected_both_wrong": 2485 + }, + { + "budget": 0.44, + "threshold": -0.4336964190006256, + "operator": "ge", + "actual_fraction": 0.39049428579530954, + "metric": 0.42007203017326944, + "rescued": 6128, + "harmed": 726, + "missed_rescue": 12692, + "avoided_harm": 1244, + "selected_both_correct": 4373, + "selected_both_wrong": 2543 + }, + { + "budget": 0.45, + "threshold": -0.4373021125793457, + "operator": "ge", + "actual_fraction": 0.4023764285511726, + "metric": 0.42665116410969006, + "rescued": 6384, + "harmed": 750, + "missed_rescue": 12436, + "avoided_harm": 1220, + "selected_both_correct": 4432, + "selected_both_wrong": 2623 + }, + { + "budget": 0.46, + "threshold": -0.4404309391975403, + "operator": "ge", + "actual_fraction": 0.41196154609647506, + "metric": 0.4321810396165953, + "rescued": 6599, + "harmed": 770, + "missed_rescue": 12221, + "avoided_harm": 1200, + "selected_both_correct": 4474, + "selected_both_wrong": 2684 + }, + { + "budget": 0.47, + "threshold": -0.44344601035118103, + "operator": "ge", + "actual_fraction": 0.42157502197771035, + "metric": 0.43782434846723195, + "rescued": 6813, + "harmed": 785, + "missed_rescue": 12007, + "avoided_harm": 1185, + "selected_both_correct": 4509, + "selected_both_wrong": 2759 + }, + { + "budget": 0.48, + "threshold": -0.4465356171131134, + "operator": "ge", + "actual_fraction": 0.43184073958540115, + "metric": 0.4434960156538014, + "rescued": 7036, + "harmed": 808, + "missed_rescue": 11784, + "avoided_harm": 1162, + "selected_both_correct": 4557, + "selected_both_wrong": 2827 + }, + { + "budget": 0.49, + "threshold": -0.44965147972106934, + "operator": "ge", + "actual_fraction": 0.44165272381816634, + "metric": 0.4488557411451096, + "rescued": 7253, + "harmed": 836, + "missed_rescue": 11567, + "avoided_harm": 1134, + "selected_both_correct": 4604, + "selected_both_wrong": 2881 + }, + { + "budget": 0.5, + "threshold": -0.453143447637558, + "operator": "ge", + "actual_fraction": 0.452315458128917, + "metric": 0.4548393500269404, + "rescued": 7483, + "harmed": 855, + "missed_rescue": 11337, + "avoided_harm": 1115, + "selected_both_correct": 4651, + "selected_both_wrong": 2961 + }, + { + "budget": 0.51, + "threshold": -0.4565357565879822, + "operator": "ge", + "actual_fraction": 0.4631483424552647, + "metric": 0.4610781839321669, + "rescued": 7721, + "harmed": 873, + "missed_rescue": 11099, + "avoided_harm": 1097, + "selected_both_correct": 4690, + "selected_both_wrong": 3048 + }, + { + "budget": 0.52, + "threshold": -0.45961302518844604, + "operator": "ge", + "actual_fraction": 0.47176927657885037, + "metric": 0.46538865099395965, + "rescued": 7891, + "harmed": 891, + "missed_rescue": 10929, + "avoided_harm": 1079, + "selected_both_correct": 4737, + "selected_both_wrong": 3117 + }, + { + "budget": 0.53, + "threshold": -0.46368208527565, + "operator": "ge", + "actual_fraction": 0.4840484360377733, + "metric": 0.4715707682273204, + "rescued": 8142, + "harmed": 924, + "missed_rescue": 10678, + "avoided_harm": 1046, + "selected_both_correct": 4791, + "selected_both_wrong": 3212 + }, + { + "budget": 0.54, + "threshold": -0.46793216466903687, + "operator": "ge", + "actual_fraction": 0.495079828715651, + "metric": 0.47809318549187535, + "rescued": 8393, + "harmed": 945, + "missed_rescue": 10427, + "avoided_harm": 1025, + "selected_both_correct": 4846, + "selected_both_wrong": 3274 + }, + { + "budget": 0.55, + "threshold": -0.4718930721282959, + "operator": "ge", + "actual_fraction": 0.5065365964325214, + "metric": 0.48393500269404194, + "rescued": 8631, + "harmed": 977, + "missed_rescue": 10189, + "avoided_harm": 993, + "selected_both_correct": 4900, + "selected_both_wrong": 3354 + }, + { + "budget": 0.56, + "threshold": -0.47629430890083313, + "operator": "ge", + "actual_fraction": 0.5183620225165188, + "metric": 0.4900887615914698, + "rescued": 8881, + "harmed": 1010, + "missed_rescue": 9939, + "avoided_harm": 960, + "selected_both_correct": 4950, + "selected_both_wrong": 3438 + }, + { + "budget": 0.57, + "threshold": -0.4799485504627228, + "operator": "ge", + "actual_fraction": 0.5273232566712985, + "metric": 0.4943708703173298, + "rescued": 9063, + "harmed": 1041, + "missed_rescue": 9757, + "avoided_harm": 929, + "selected_both_correct": 4989, + "selected_both_wrong": 3502 + }, + { + "budget": 0.58, + "threshold": -0.4838995635509491, + "operator": "ge", + "actual_fraction": 0.5381561409976463, + "metric": 0.5008082125740861, + "rescued": 9310, + "harmed": 1061, + "missed_rescue": 9510, + "avoided_harm": 909, + "selected_both_correct": 5038, + "selected_both_wrong": 3568 + }, + { + "budget": 0.59, + "threshold": -0.4886912405490875, + "operator": "ge", + "actual_fraction": 0.5503502254487707, + "metric": 0.5080395882369623, + "rescued": 9590, + "harmed": 1086, + "missed_rescue": 9230, + "avoided_harm": 884, + "selected_both_correct": 5080, + "selected_both_wrong": 3651 + }, + { + "budget": 0.6, + "threshold": -0.49332454800605774, + "operator": "ge", + "actual_fraction": 0.5619487848453053, + "metric": 0.5147321555171143, + "rescued": 9849, + "harmed": 1109, + "missed_rescue": 8971, + "avoided_harm": 861, + "selected_both_correct": 5127, + "selected_both_wrong": 3731 + }, + { + "budget": 0.61, + "threshold": -0.49750789999961853, + "operator": "ge", + "actual_fraction": 0.5722712191248618, + "metric": 0.5204038227036838, + "rescued": 10085, + "harmed": 1145, + "missed_rescue": 8735, + "avoided_harm": 825, + "selected_both_correct": 5165, + "selected_both_wrong": 3785 + }, + { + "budget": 0.62, + "threshold": -0.5024909377098083, + "operator": "ge", + "actual_fraction": 0.5818563366701642, + "metric": 0.5257635481949919, + "rescued": 10296, + "harmed": 1167, + "missed_rescue": 8524, + "avoided_harm": 803, + "selected_both_correct": 5206, + "selected_both_wrong": 3849 + }, + { + "budget": 0.63, + "threshold": -0.5067122578620911, + "operator": "ge", + "actual_fraction": 0.590817570824944, + "metric": 0.5306128236395088, + "rescued": 10487, + "harmed": 1187, + "missed_rescue": 8333, + "avoided_harm": 783, + "selected_both_correct": 5248, + "selected_both_wrong": 3912 + }, + { + "budget": 0.64, + "threshold": -0.512323796749115, + "operator": "ge", + "actual_fraction": 0.604202705385248, + "metric": 0.5381277826617135, + "rescued": 10781, + "harmed": 1216, + "missed_rescue": 8039, + "avoided_harm": 754, + "selected_both_correct": 5304, + "selected_both_wrong": 4005 + }, + { + "budget": 0.65, + "threshold": -0.5170255303382874, + "operator": "ge", + "actual_fraction": 0.6130505061962964, + "metric": 0.5431472081218274, + "rescued": 10982, + "harmed": 1240, + "missed_rescue": 7838, + "avoided_harm": 730, + "selected_both_correct": 5343, + "selected_both_wrong": 4053 + }, + { + "budget": 0.66, + "threshold": -0.5215396881103516, + "operator": "ge", + "actual_fraction": 0.6222102487026061, + "metric": 0.548421858605337, + "rescued": 11191, + "harmed": 1263, + "missed_rescue": 7629, + "avoided_harm": 707, + "selected_both_correct": 5388, + "selected_both_wrong": 4099 + }, + { + "budget": 0.67, + "threshold": -0.5264327526092529, + "operator": "ge", + "actual_fraction": 0.6320505912713043, + "metric": 0.552902475682727, + "rescued": 11382, + "harmed": 1296, + "missed_rescue": 7438, + "avoided_harm": 674, + "selected_both_correct": 5428, + "selected_both_wrong": 4182 + }, + { + "budget": 0.68, + "threshold": -0.5312389135360718, + "operator": "ge", + "actual_fraction": 0.6424297422227264, + "metric": 0.5588010095567592, + "rescued": 11615, + "harmed": 1321, + "missed_rescue": 7205, + "avoided_harm": 649, + "selected_both_correct": 5473, + "selected_both_wrong": 4245 + }, + { + "budget": 0.69, + "threshold": -0.5368650555610657, + "operator": "ge", + "actual_fraction": 0.6538865099395967, + "metric": 0.5642174517199331, + "rescued": 11846, + "harmed": 1361, + "missed_rescue": 6974, + "avoided_harm": 609, + "selected_both_correct": 5528, + "selected_both_wrong": 4323 + }, + { + "budget": 0.7, + "threshold": -0.5431069135665894, + "operator": "ge", + "actual_fraction": 0.6647761109378102, + "metric": 0.5704562856251595, + "rescued": 12087, + "harmed": 1382, + "missed_rescue": 6733, + "avoided_harm": 588, + "selected_both_correct": 5573, + "selected_both_wrong": 4400 + }, + { + "budget": 0.71, + "threshold": -0.5487136244773865, + "operator": "ge", + "actual_fraction": 0.6740492867878513, + "metric": 0.5753339194056093, + "rescued": 12285, + "harmed": 1408, + "missed_rescue": 6535, + "avoided_harm": 562, + "selected_both_correct": 5609, + "selected_both_wrong": 4467 + }, + { + "budget": 0.72, + "threshold": -0.5549414753913879, + "operator": "ge", + "actual_fraction": 0.6839179876924822, + "metric": 0.5810906615999774, + "rescued": 12503, + "harmed": 1423, + "missed_rescue": 6317, + "avoided_harm": 547, + "selected_both_correct": 5664, + "selected_both_wrong": 4527 + }, + { + "budget": 0.73, + "threshold": -0.5604677200317383, + "operator": "ge", + "actual_fraction": 0.6935314635737175, + "metric": 0.5866205371068826, + "rescued": 12724, + "harmed": 1449, + "missed_rescue": 6096, + "avoided_harm": 521, + "selected_both_correct": 5702, + "selected_both_wrong": 4581 + }, + { + "budget": 0.74, + "threshold": -0.5666090250015259, + "operator": "ge", + "actual_fraction": 0.7039673311970054, + "metric": 0.5924623543090491, + "rescued": 12953, + "harmed": 1472, + "missed_rescue": 5867, + "avoided_harm": 498, + "selected_both_correct": 5739, + "selected_both_wrong": 4660 + }, + { + "budget": 0.75, + "threshold": -0.574927568435669, + "operator": "ge", + "actual_fraction": 0.7186853075461532, + "metric": 0.6001758216827836, + "rescued": 13270, + "harmed": 1517, + "missed_rescue": 5550, + "avoided_harm": 453, + "selected_both_correct": 5813, + "selected_both_wrong": 4743 + }, + { + "budget": 0.76, + "threshold": -0.5812664031982422, + "operator": "ge", + "actual_fraction": 0.7283555000992542, + "metric": 0.6051668888069648, + "rescued": 13473, + "harmed": 1544, + "missed_rescue": 5347, + "avoided_harm": 426, + "selected_both_correct": 5862, + "selected_both_wrong": 4805 + }, + { + "budget": 0.77, + "threshold": -0.5883752107620239, + "operator": "ge", + "actual_fraction": 0.7390465927459376, + "metric": 0.6112072143606613, + "rescued": 13710, + "harmed": 1568, + "missed_rescue": 5110, + "avoided_harm": 402, + "selected_both_correct": 5909, + "selected_both_wrong": 4874 + }, + { + "budget": 0.78, + "threshold": -0.5972744226455688, + "operator": "ge", + "actual_fraction": 0.752176502282846, + "metric": 0.618722173382866, + "rescued": 14008, + "harmed": 1601, + "missed_rescue": 4812, + "avoided_harm": 369, + "selected_both_correct": 5959, + "selected_both_wrong": 4956 + }, + { + "budget": 0.79, + "threshold": -0.6058458089828491, + "operator": "ge", + "actual_fraction": 0.7635765533278507, + "metric": 0.6246490655928311, + "rescued": 14250, + "harmed": 1634, + "missed_rescue": 4570, + "avoided_harm": 336, + "selected_both_correct": 6010, + "selected_both_wrong": 5032 + }, + { + "budget": 0.8, + "threshold": -0.6140543818473816, + "operator": "ge", + "actual_fraction": 0.7745228709979298, + "metric": 0.6308878994980575, + "rescued": 14487, + "harmed": 1651, + "missed_rescue": 4333, + "avoided_harm": 319, + "selected_both_correct": 6062, + "selected_both_wrong": 5112 + }, + { + "budget": 0.81, + "threshold": -0.6230268478393555, + "operator": "ge", + "actual_fraction": 0.7864050137537929, + "metric": 0.6366730000283584, + "rescued": 14732, + "harmed": 1692, + "missed_rescue": 4088, + "avoided_harm": 278, + "selected_both_correct": 6105, + "selected_both_wrong": 5202 + }, + { + "budget": 0.82, + "threshold": -0.6330366134643555, + "operator": "ge", + "actual_fraction": 0.798287156509656, + "metric": 0.6438476590193687, + "rescued": 15012, + "harmed": 1719, + "missed_rescue": 3808, + "avoided_harm": 251, + "selected_both_correct": 6148, + "selected_both_wrong": 5271 + }, + { + "budget": 0.83, + "threshold": -0.6429333686828613, + "operator": "ge", + "actual_fraction": 0.8092334741797351, + "metric": 0.6499730595808638, + "rescued": 15248, + "harmed": 1739, + "missed_rescue": 3572, + "avoided_harm": 231, + "selected_both_correct": 6207, + "selected_both_wrong": 5342 + }, + { + "budget": 0.84, + "threshold": -0.6513019800186157, + "operator": "ge", + "actual_fraction": 0.8177126166236566, + "metric": 0.6548223350253807, + "rescued": 15437, + "harmed": 1757, + "missed_rescue": 3383, + "avoided_harm": 213, + "selected_both_correct": 6242, + "selected_both_wrong": 5399 + }, + { + "budget": 0.85, + "threshold": -0.6624053120613098, + "operator": "ge", + "actual_fraction": 0.8288858009811985, + "metric": 0.6607492272353458, + "rescued": 15678, + "harmed": 1789, + "missed_rescue": 3142, + "avoided_harm": 181, + "selected_both_correct": 6295, + "selected_both_wrong": 5467 + }, + { + "budget": 0.86, + "threshold": -0.6759263277053833, + "operator": "ge", + "actual_fraction": 0.8421291438618382, + "metric": 0.6682925445934833, + "rescued": 15980, + "harmed": 1825, + "missed_rescue": 2840, + "avoided_harm": 145, + "selected_both_correct": 6343, + "selected_both_wrong": 5548 + }, + { + "budget": 0.87, + "threshold": -0.6904275417327881, + "operator": "ge", + "actual_fraction": 0.8543799449848283, + "metric": 0.6742194368034484, + "rescued": 16226, + "harmed": 1862, + "missed_rescue": 2594, + "avoided_harm": 108, + "selected_both_correct": 6386, + "selected_both_wrong": 5654 + }, + { + "budget": 0.88, + "threshold": -0.7045450210571289, + "operator": "ge", + "actual_fraction": 0.8657516376939001, + "metric": 0.6793806539432267, + "rescued": 16441, + "harmed": 1895, + "missed_rescue": 2379, + "avoided_harm": 75, + "selected_both_correct": 6451, + "selected_both_wrong": 5742 + }, + { + "budget": 0.89, + "threshold": -0.7205488681793213, + "operator": "ge", + "actual_fraction": 0.8767830303717777, + "metric": 0.6845985877548706, + "rescued": 16636, + "harmed": 1906, + "missed_rescue": 2184, + "avoided_harm": 64, + "selected_both_correct": 6534, + "selected_both_wrong": 5842 + }, + { + "budget": 0.9, + "threshold": -0.7357243299484253, + "operator": "ge", + "actual_fraction": 0.8868785979638715, + "metric": 0.6893911465275218, + "rescued": 16811, + "harmed": 1912, + "missed_rescue": 2009, + "avoided_harm": 58, + "selected_both_correct": 6611, + "selected_both_wrong": 5940 + }, + { + "budget": 0.91, + "threshold": -0.7513164281845093, + "operator": "ge", + "actual_fraction": 0.8974562572668235, + "metric": 0.6947508720188299, + "rescued": 17004, + "harmed": 1916, + "missed_rescue": 1816, + "avoided_harm": 54, + "selected_both_correct": 6695, + "selected_both_wrong": 6032 + }, + { + "budget": 0.92, + "threshold": -0.7671180963516235, + "operator": "ge", + "actual_fraction": 0.9084025749369027, + "metric": 0.7000255225023395, + "rescued": 17196, + "harmed": 1922, + "missed_rescue": 1624, + "avoided_harm": 48, + "selected_both_correct": 6770, + "selected_both_wrong": 6145 + }, + { + "budget": 0.93, + "threshold": -0.7833515405654907, + "operator": "ge", + "actual_fraction": 0.9181011258259365, + "metric": 0.704959872954655, + "rescued": 17376, + "harmed": 1928, + "missed_rescue": 1444, + "avoided_harm": 42, + "selected_both_correct": 6846, + "selected_both_wrong": 6225 + }, + { + "budget": 0.94, + "threshold": -0.8013419508934021, + "operator": "ge", + "actual_fraction": 0.9289623684882171, + "metric": 0.710546465133426, + "rescued": 17577, + "harmed": 1932, + "missed_rescue": 1243, + "avoided_harm": 38, + "selected_both_correct": 6926, + "selected_both_wrong": 6323 + }, + { + "budget": 0.95, + "threshold": -0.8188987374305725, + "operator": "ge", + "actual_fraction": 0.9394549527833707, + "metric": 0.7157360406091371, + "rescued": 17763, + "harmed": 1935, + "missed_rescue": 1057, + "avoided_harm": 35, + "selected_both_correct": 7010, + "selected_both_wrong": 6420 + }, + { + "budget": 0.96, + "threshold": -0.8395025134086609, + "operator": "ge", + "actual_fraction": 0.9509684371721068, + "metric": 0.7211808411082438, + "rescued": 17963, + "harmed": 1943, + "missed_rescue": 857, + "avoided_harm": 27, + "selected_both_correct": 7084, + "selected_both_wrong": 6544 + }, + { + "budget": 0.97, + "threshold": -0.8628664612770081, + "operator": "ge", + "actual_fraction": 0.9640699883730822, + "metric": 0.727476391685336, + "rescued": 18191, + "harmed": 1949, + "missed_rescue": 629, + "avoided_harm": 21, + "selected_both_correct": 7177, + "selected_both_wrong": 6679 + }, + { + "budget": 0.98, + "threshold": -0.8857419490814209, + "operator": "ge", + "actual_fraction": 0.9760372061367439, + "metric": 0.7329779088563083, + "rescued": 18395, + "harmed": 1959, + "missed_rescue": 425, + "avoided_harm": 11, + "selected_both_correct": 7278, + "selected_both_wrong": 6786 + }, + { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9873805405098829, + "metric": 0.7387062927147435, + "rescued": 18604, + "harmed": 1966, + "missed_rescue": 216, + "avoided_harm": 4, + "selected_both_correct": 7360, + "selected_both_wrong": 6888 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.7447182599325072, + "rescued": 18820, + "harmed": 1970, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 7441, + "selected_both_wrong": 7032 + } + ] + }, + "learned": { + "selected_budget_index": 42, + "validation_area_0_25": 0.669126431934781, + "validation_point": { + "budget": 0.42, + "threshold": 5.466009616851807, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.8153067678684377, + "rescued": 3932, + "harmed": 307, + "missed_rescue": 694, + "avoided_harm": 826, + "selected_both_correct": 581, + "selected_both_wrong": 1156 + }, + "test_point": { + "budget": 0.42, + "threshold": 5.466009616851807, + "operator": "ge", + "actual_fraction": 0.5745682443354224, + "metric": 0.6610044522587415, + "rescued": 14636, + "harmed": 738, + "missed_rescue": 4184, + "avoided_harm": 1232, + "selected_both_correct": 725, + "selected_both_wrong": 4162 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.5605453650994447, + "rescued": 0, + "harmed": 0, + "missed_rescue": 4626, + "avoided_harm": 1133, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 21.90387725830078, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.5700330311336004, + "rescued": 135, + "harmed": 0, + "missed_rescue": 4491, + "avoided_harm": 1133, + "selected_both_correct": 0, + "selected_both_wrong": 7 + }, + { + "budget": 0.02, + "threshold": 20.418020248413086, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.5793098601447747, + "rescued": 267, + "harmed": 0, + "missed_rescue": 4359, + "avoided_harm": 1133, + "selected_both_correct": 0, + "selected_both_wrong": 17 + }, + { + "budget": 0.03, + "threshold": 19.33527946472168, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.5887272471712699, + "rescued": 401, + "harmed": 0, + "missed_rescue": 4225, + "avoided_harm": 1133, + "selected_both_correct": 0, + "selected_both_wrong": 25 + }, + { + "budget": 0.04, + "threshold": 18.620174407958984, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.598285192213086, + "rescued": 537, + "harmed": 0, + "missed_rescue": 4089, + "avoided_harm": 1133, + "selected_both_correct": 0, + "selected_both_wrong": 32 + }, + { + "budget": 0.05, + "threshold": 17.99974250793457, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.6074917422165999, + "rescued": 668, + "harmed": 0, + "missed_rescue": 3958, + "avoided_harm": 1133, + "selected_both_correct": 0, + "selected_both_wrong": 43 + }, + { + "budget": 0.06, + "threshold": 17.459346771240234, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.6163468971818118, + "rescued": 794, + "harmed": 0, + "missed_rescue": 3832, + "avoided_harm": 1133, + "selected_both_correct": 0, + "selected_both_wrong": 59 + }, + { + "budget": 0.07, + "threshold": 16.8415470123291, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.6256940052006466, + "rescued": 927, + "harmed": 0, + "missed_rescue": 3699, + "avoided_harm": 1133, + "selected_both_correct": 0, + "selected_both_wrong": 69 + }, + { + "budget": 0.08, + "threshold": 16.37276268005371, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.6346897181811793, + "rescued": 1055, + "harmed": 0, + "missed_rescue": 3571, + "avoided_harm": 1133, + "selected_both_correct": 0, + "selected_both_wrong": 83 + }, + { + "budget": 0.09, + "threshold": 15.89695930480957, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.6431934781080891, + "rescued": 1176, + "harmed": 0, + "missed_rescue": 3450, + "avoided_harm": 1133, + "selected_both_correct": 1, + "selected_both_wrong": 103 + }, + { + "budget": 0.1, + "threshold": 15.354147911071777, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.6516972380349989, + "rescued": 1297, + "harmed": 0, + "missed_rescue": 3329, + "avoided_harm": 1133, + "selected_both_correct": 1, + "selected_both_wrong": 124 + }, + { + "budget": 0.11, + "threshold": 14.913481712341309, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.6605523930002108, + "rescued": 1423, + "harmed": 0, + "missed_rescue": 3203, + "avoided_harm": 1133, + "selected_both_correct": 4, + "selected_both_wrong": 138 + }, + { + "budget": 0.12, + "threshold": 14.473790168762207, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.669126431934781, + "rescued": 1545, + "harmed": 0, + "missed_rescue": 3081, + "avoided_harm": 1133, + "selected_both_correct": 5, + "selected_both_wrong": 157 + }, + { + "budget": 0.13, + "threshold": 14.001596450805664, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.6774896338463701, + "rescued": 1664, + "harmed": 0, + "missed_rescue": 2962, + "avoided_harm": 1133, + "selected_both_correct": 6, + "selected_both_wrong": 179 + }, + { + "budget": 0.14, + "threshold": 13.631171226501465, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.6852203246890154, + "rescued": 1774, + "harmed": 0, + "missed_rescue": 2852, + "avoided_harm": 1133, + "selected_both_correct": 6, + "selected_both_wrong": 212 + }, + { + "budget": 0.15, + "threshold": 13.249896049499512, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.6929510155316607, + "rescued": 1885, + "harmed": 1, + "missed_rescue": 2741, + "avoided_harm": 1132, + "selected_both_correct": 6, + "selected_both_wrong": 242 + }, + { + "budget": 0.16, + "threshold": 12.844018936157227, + "operator": "ge", + "actual_fraction": 0.15995502143509735, + "metric": 0.7004708693513247, + "rescued": 1993, + "harmed": 2, + "missed_rescue": 2633, + "avoided_harm": 1131, + "selected_both_correct": 7, + "selected_both_wrong": 274 + }, + { + "budget": 0.17, + "threshold": 12.422953605651855, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.7083421182092909, + "rescued": 2106, + "harmed": 3, + "missed_rescue": 2520, + "avoided_harm": 1130, + "selected_both_correct": 8, + "selected_both_wrong": 301 + }, + { + "budget": 0.18, + "threshold": 12.03639030456543, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.7161430880595966, + "rescued": 2217, + "harmed": 3, + "missed_rescue": 2409, + "avoided_harm": 1130, + "selected_both_correct": 9, + "selected_both_wrong": 332 + }, + { + "budget": 0.19, + "threshold": 11.710915565490723, + "operator": "ge", + "actual_fraction": 0.18996415770609318, + "metric": 0.7239440579099023, + "rescued": 2330, + "harmed": 5, + "missed_rescue": 2296, + "avoided_harm": 1128, + "selected_both_correct": 10, + "selected_both_wrong": 358 + }, + { + "budget": 0.2, + "threshold": 11.339479446411133, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.7310422376836039, + "rescued": 2432, + "harmed": 6, + "missed_rescue": 2194, + "avoided_harm": 1127, + "selected_both_correct": 11, + "selected_both_wrong": 396 + }, + { + "budget": 0.21, + "threshold": 11.019915580749512, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.7386323705109283, + "rescued": 2542, + "harmed": 8, + "missed_rescue": 2084, + "avoided_harm": 1125, + "selected_both_correct": 13, + "selected_both_wrong": 425 + }, + { + "budget": 0.22, + "threshold": 10.663690567016602, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.7445358071544029, + "rescued": 2629, + "harmed": 11, + "missed_rescue": 1997, + "avoided_harm": 1122, + "selected_both_correct": 15, + "selected_both_wrong": 475 + }, + { + "budget": 0.23, + "threshold": 10.3711576461792, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.75086091784384, + "rescued": 2724, + "harmed": 16, + "missed_rescue": 1902, + "avoided_harm": 1117, + "selected_both_correct": 16, + "selected_both_wrong": 516 + }, + { + "budget": 0.24, + "threshold": 10.053378105163574, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.7579590976175417, + "rescued": 2826, + "harmed": 17, + "missed_rescue": 1800, + "avoided_harm": 1116, + "selected_both_correct": 18, + "selected_both_wrong": 553 + }, + { + "budget": 0.25, + "threshold": 9.729846000671387, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.7643544873146391, + "rescued": 2921, + "harmed": 21, + "missed_rescue": 1705, + "avoided_harm": 1112, + "selected_both_correct": 24, + "selected_both_wrong": 591 + }, + { + "budget": 0.26, + "threshold": 9.389237403869629, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.7706093189964157, + "rescued": 3016, + "harmed": 27, + "missed_rescue": 1610, + "avoided_harm": 1106, + "selected_both_correct": 28, + "selected_both_wrong": 628 + }, + { + "budget": 0.27, + "threshold": 9.115911483764648, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.7763721976245695, + "rescued": 3106, + "harmed": 35, + "missed_rescue": 1520, + "avoided_harm": 1098, + "selected_both_correct": 36, + "selected_both_wrong": 664 + }, + { + "budget": 0.28, + "threshold": 8.828272819519043, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.7810106121301568, + "rescued": 3180, + "harmed": 43, + "missed_rescue": 1446, + "avoided_harm": 1090, + "selected_both_correct": 47, + "selected_both_wrong": 714 + }, + { + "budget": 0.29, + "threshold": 8.522764205932617, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.786351816712348, + "rescued": 3264, + "harmed": 51, + "missed_rescue": 1362, + "avoided_harm": 1082, + "selected_both_correct": 56, + "selected_both_wrong": 755 + }, + { + "budget": 0.3, + "threshold": 8.2249174118042, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.7906388361796332, + "rescued": 3340, + "harmed": 66, + "missed_rescue": 1286, + "avoided_harm": 1067, + "selected_both_correct": 78, + "selected_both_wrong": 784 + }, + { + "budget": 0.31, + "threshold": 7.953249454498291, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.7935202754937101, + "rescued": 3399, + "harmed": 84, + "missed_rescue": 1227, + "avoided_harm": 1049, + "selected_both_correct": 109, + "selected_both_wrong": 818 + }, + { + "budget": 0.32, + "threshold": 7.666618347167969, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.7965422728231077, + "rescued": 3457, + "harmed": 99, + "missed_rescue": 1169, + "avoided_harm": 1034, + "selected_both_correct": 136, + "selected_both_wrong": 861 + }, + { + "budget": 0.33, + "threshold": 7.3823370933532715, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.7997048281678263, + "rescued": 3516, + "harmed": 113, + "missed_rescue": 1110, + "avoided_harm": 1020, + "selected_both_correct": 158, + "selected_both_wrong": 908 + }, + { + "budget": 0.34, + "threshold": 7.096353530883789, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.8027971045048844, + "rescued": 3573, + "harmed": 126, + "missed_rescue": 1053, + "avoided_harm": 1007, + "selected_both_correct": 194, + "selected_both_wrong": 944 + }, + { + "budget": 0.35, + "threshold": 6.865462779998779, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.803570173589149, + "rescued": 3613, + "harmed": 155, + "missed_rescue": 1013, + "avoided_harm": 978, + "selected_both_correct": 228, + "selected_both_wrong": 984 + }, + { + "budget": 0.36, + "threshold": 6.669365882873535, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.8060299388572634, + "rescued": 3668, + "harmed": 175, + "missed_rescue": 958, + "avoided_harm": 958, + "selected_both_correct": 269, + "selected_both_wrong": 1010 + }, + { + "budget": 0.37, + "threshold": 6.469961166381836, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.8081383090870757, + "rescued": 3717, + "harmed": 194, + "missed_rescue": 909, + "avoided_harm": 939, + "selected_both_correct": 317, + "selected_both_wrong": 1036 + }, + { + "budget": 0.38, + "threshold": 6.239957809448242, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.8097547262632652, + "rescued": 3767, + "harmed": 221, + "missed_rescue": 859, + "avoided_harm": 912, + "selected_both_correct": 359, + "selected_both_wrong": 1060 + }, + { + "budget": 0.39, + "threshold": 6.02949333190918, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.8108791903858318, + "rescued": 3807, + "harmed": 245, + "missed_rescue": 819, + "avoided_harm": 888, + "selected_both_correct": 414, + "selected_both_wrong": 1083 + }, + { + "budget": 0.4, + "threshold": 5.813343048095703, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.8127767235926628, + "rescued": 3852, + "harmed": 263, + "missed_rescue": 774, + "avoided_harm": 870, + "selected_both_correct": 470, + "selected_both_wrong": 1106 + }, + { + "budget": 0.41, + "threshold": 5.622989177703857, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.8146039777918336, + "rescued": 3895, + "harmed": 280, + "missed_rescue": 731, + "avoided_harm": 853, + "selected_both_correct": 524, + "selected_both_wrong": 1134 + }, + { + "budget": 0.42, + "threshold": 5.466009616851807, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.8153067678684377, + "rescued": 3932, + "harmed": 307, + "missed_rescue": 694, + "avoided_harm": 826, + "selected_both_correct": 581, + "selected_both_wrong": 1156 + }, + { + "budget": 0.43, + "threshold": 5.273525238037109, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.8150256518377961, + "rescued": 3961, + "harmed": 340, + "missed_rescue": 665, + "avoided_harm": 793, + "selected_both_correct": 636, + "selected_both_wrong": 1181 + }, + { + "budget": 0.44, + "threshold": 5.097577095031738, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.8152364888607773, + "rescued": 3992, + "harmed": 368, + "missed_rescue": 634, + "avoided_harm": 765, + "selected_both_correct": 700, + "selected_both_wrong": 1200 + }, + { + "budget": 0.45, + "threshold": 4.9260430335998535, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.8149553728301356, + "rescued": 4013, + "harmed": 393, + "missed_rescue": 613, + "avoided_harm": 740, + "selected_both_correct": 779, + "selected_both_wrong": 1218 + }, + { + "budget": 0.46, + "threshold": 4.770597457885742, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.8139714667228899, + "rescued": 4034, + "harmed": 428, + "missed_rescue": 592, + "avoided_harm": 705, + "selected_both_correct": 854, + "selected_both_wrong": 1229 + }, + { + "budget": 0.47, + "threshold": 4.615985870361328, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.8133389556539462, + "rescued": 4056, + "harmed": 459, + "missed_rescue": 570, + "avoided_harm": 674, + "selected_both_correct": 927, + "selected_both_wrong": 1245 + }, + { + "budget": 0.48, + "threshold": 4.460105895996094, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.8127767235926628, + "rescued": 4075, + "harmed": 486, + "missed_rescue": 551, + "avoided_harm": 647, + "selected_both_correct": 1003, + "selected_both_wrong": 1265 + }, + { + "budget": 0.49, + "threshold": 4.310606479644775, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.81228477053904, + "rescued": 4095, + "harmed": 513, + "missed_rescue": 531, + "avoided_harm": 620, + "selected_both_correct": 1083, + "selected_both_wrong": 1281 + }, + { + "budget": 0.5, + "threshold": 4.1707234382629395, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.8116522594700962, + "rescued": 4113, + "harmed": 540, + "missed_rescue": 513, + "avoided_harm": 593, + "selected_both_correct": 1166, + "selected_both_wrong": 1295 + }, + { + "budget": 0.51, + "threshold": 4.034754753112793, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.8107386323705109, + "rescued": 4134, + "harmed": 574, + "missed_rescue": 492, + "avoided_harm": 559, + "selected_both_correct": 1249, + "selected_both_wrong": 1299 + }, + { + "budget": 0.52, + "threshold": 3.9028453826904297, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.8098250052709256, + "rescued": 4153, + "harmed": 606, + "missed_rescue": 473, + "avoided_harm": 527, + "selected_both_correct": 1327, + "selected_both_wrong": 1313 + }, + { + "budget": 0.53, + "threshold": 3.742058277130127, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.8096844472556047, + "rescued": 4176, + "harmed": 631, + "missed_rescue": 450, + "avoided_harm": 502, + "selected_both_correct": 1412, + "selected_both_wrong": 1322 + }, + { + "budget": 0.54, + "threshold": 3.5907771587371826, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.8097547262632652, + "rescued": 4198, + "harmed": 652, + "missed_rescue": 428, + "avoided_harm": 481, + "selected_both_correct": 1495, + "selected_both_wrong": 1338 + }, + { + "budget": 0.55, + "threshold": 3.454869270324707, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.8104575163398693, + "rescued": 4224, + "harmed": 668, + "missed_rescue": 402, + "avoided_harm": 465, + "selected_both_correct": 1585, + "selected_both_wrong": 1348 + }, + { + "budget": 0.56, + "threshold": 3.3179640769958496, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.8104575163398693, + "rescued": 4244, + "harmed": 688, + "missed_rescue": 382, + "avoided_harm": 445, + "selected_both_correct": 1680, + "selected_both_wrong": 1356 + }, + { + "budget": 0.57, + "threshold": 3.192162036895752, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.8103872373322089, + "rescued": 4261, + "harmed": 706, + "missed_rescue": 365, + "avoided_harm": 427, + "selected_both_correct": 1778, + "selected_both_wrong": 1365 + }, + { + "budget": 0.58, + "threshold": 3.0637471675872803, + "operator": "ge", + "actual_fraction": 0.5799423712137185, + "metric": 0.8091924942019819, + "rescued": 4271, + "harmed": 733, + "missed_rescue": 355, + "avoided_harm": 400, + "selected_both_correct": 1874, + "selected_both_wrong": 1374 + }, + { + "budget": 0.59, + "threshold": 2.9273691177368164, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.8094033312249631, + "rescued": 4293, + "harmed": 752, + "missed_rescue": 333, + "avoided_harm": 381, + "selected_both_correct": 1970, + "selected_both_wrong": 1380 + }, + { + "budget": 0.6, + "threshold": 2.7924067974090576, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.8092627732096422, + "rescued": 4305, + "harmed": 766, + "missed_rescue": 321, + "avoided_harm": 367, + "selected_both_correct": 2074, + "selected_both_wrong": 1392 + }, + { + "budget": 0.61, + "threshold": 2.633131504058838, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.8091924942019819, + "rescued": 4325, + "harmed": 787, + "missed_rescue": 301, + "avoided_harm": 346, + "selected_both_correct": 2169, + "selected_both_wrong": 1398 + }, + { + "budget": 0.62, + "threshold": 2.5002918243408203, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.8097547262632652, + "rescued": 4350, + "harmed": 804, + "missed_rescue": 276, + "avoided_harm": 329, + "selected_both_correct": 2251, + "selected_both_wrong": 1416 + }, + { + "budget": 0.63, + "threshold": 2.3700814247131348, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.8091924942019819, + "rescued": 4362, + "harmed": 824, + "missed_rescue": 264, + "avoided_harm": 309, + "selected_both_correct": 2352, + "selected_both_wrong": 1426 + }, + { + "budget": 0.64, + "threshold": 2.223950147628784, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.8083491461100569, + "rescued": 4376, + "harmed": 850, + "missed_rescue": 250, + "avoided_harm": 283, + "selected_both_correct": 2441, + "selected_both_wrong": 1439 + }, + { + "budget": 0.65, + "threshold": 2.0700783729553223, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.8088410991636799, + "rescued": 4399, + "harmed": 866, + "missed_rescue": 227, + "avoided_harm": 267, + "selected_both_correct": 2527, + "selected_both_wrong": 1456 + }, + { + "budget": 0.66, + "threshold": 1.9317810535430908, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.809051936186661, + "rescued": 4417, + "harmed": 881, + "missed_rescue": 209, + "avoided_harm": 252, + "selected_both_correct": 2625, + "selected_both_wrong": 1468 + }, + { + "budget": 0.67, + "threshold": 1.7615712881088257, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.808700541148359, + "rescued": 4431, + "harmed": 900, + "missed_rescue": 195, + "avoided_harm": 233, + "selected_both_correct": 2725, + "selected_both_wrong": 1477 + }, + { + "budget": 0.68, + "threshold": 1.6136956214904785, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.8089113781713402, + "rescued": 4449, + "harmed": 915, + "missed_rescue": 177, + "avoided_harm": 218, + "selected_both_correct": 2818, + "selected_both_wrong": 1493 + }, + { + "budget": 0.69, + "threshold": 1.4685550928115845, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.8099655632862464, + "rescued": 4472, + "harmed": 923, + "missed_rescue": 154, + "avoided_harm": 210, + "selected_both_correct": 2920, + "selected_both_wrong": 1503 + }, + { + "budget": 0.7, + "threshold": 1.327118992805481, + "operator": "ge", + "actual_fraction": 0.6999789162977019, + "metric": 0.8107386323705109, + "rescued": 4491, + "harmed": 931, + "missed_rescue": 135, + "avoided_harm": 202, + "selected_both_correct": 3017, + "selected_both_wrong": 1521 + }, + { + "budget": 0.71, + "threshold": 1.1890794038772583, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.8115117014547755, + "rescued": 4510, + "harmed": 939, + "missed_rescue": 116, + "avoided_harm": 194, + "selected_both_correct": 3118, + "selected_both_wrong": 1535 + }, + { + "budget": 0.72, + "threshold": 1.042020320892334, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.8120739335160587, + "rescued": 4525, + "harmed": 946, + "missed_rescue": 101, + "avoided_harm": 187, + "selected_both_correct": 3221, + "selected_both_wrong": 1552 + }, + { + "budget": 0.73, + "threshold": 0.8947708606719971, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.8126361655773421, + "rescued": 4544, + "harmed": 957, + "missed_rescue": 82, + "avoided_harm": 176, + "selected_both_correct": 3326, + "selected_both_wrong": 1560 + }, + { + "budget": 0.74, + "threshold": 0.7377536296844482, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.8129172816079837, + "rescued": 4557, + "harmed": 966, + "missed_rescue": 69, + "avoided_harm": 167, + "selected_both_correct": 3436, + "selected_both_wrong": 1570 + }, + { + "budget": 0.75, + "threshold": 0.5805552005767822, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.8132686766462858, + "rescued": 4571, + "harmed": 975, + "missed_rescue": 55, + "avoided_harm": 158, + "selected_both_correct": 3538, + "selected_both_wrong": 1587 + }, + { + "budget": 0.76, + "threshold": 0.41288113594055176, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.8134092346616065, + "rescued": 4580, + "harmed": 982, + "missed_rescue": 46, + "avoided_harm": 151, + "selected_both_correct": 3656, + "selected_both_wrong": 1596 + }, + { + "budget": 0.77, + "threshold": 0.23000085353851318, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.8141120247382106, + "rescued": 4594, + "harmed": 986, + "missed_rescue": 32, + "avoided_harm": 147, + "selected_both_correct": 3773, + "selected_both_wrong": 1603 + }, + { + "budget": 0.78, + "threshold": 0.06256341934204102, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.8141823037458711, + "rescued": 4603, + "harmed": 994, + "missed_rescue": 23, + "avoided_harm": 139, + "selected_both_correct": 3891, + "selected_both_wrong": 1610 + }, + { + "budget": 0.79, + "threshold": -0.12610122561454773, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.8140417457305503, + "rescued": 4610, + "harmed": 1003, + "missed_rescue": 16, + "avoided_harm": 130, + "selected_both_correct": 4014, + "selected_both_wrong": 1613 + }, + { + "budget": 0.8, + "threshold": -0.3212296962738037, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.8135497926769274, + "rescued": 4612, + "harmed": 1012, + "missed_rescue": 14, + "avoided_harm": 121, + "selected_both_correct": 4139, + "selected_both_wrong": 1620 + }, + { + "budget": 0.81, + "threshold": -0.533017635345459, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.8131983976386253, + "rescued": 4619, + "harmed": 1024, + "missed_rescue": 7, + "avoided_harm": 109, + "selected_both_correct": 4262, + "selected_both_wrong": 1620 + }, + { + "budget": 0.82, + "threshold": -0.715209424495697, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.8127064445850024, + "rescued": 4621, + "harmed": 1033, + "missed_rescue": 5, + "avoided_harm": 100, + "selected_both_correct": 4391, + "selected_both_wrong": 1622 + }, + { + "budget": 0.83, + "threshold": -0.9083468914031982, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.81228477053904, + "rescued": 4623, + "harmed": 1041, + "missed_rescue": 3, + "avoided_harm": 92, + "selected_both_correct": 4521, + "selected_both_wrong": 1625 + }, + { + "budget": 0.84, + "threshold": -1.103330135345459, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.8120739335160587, + "rescued": 4624, + "harmed": 1045, + "missed_rescue": 2, + "avoided_harm": 88, + "selected_both_correct": 4657, + "selected_both_wrong": 1626 + }, + { + "budget": 0.85, + "threshold": -1.338437557220459, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.8115117014547755, + "rescued": 4624, + "harmed": 1053, + "missed_rescue": 2, + "avoided_harm": 80, + "selected_both_correct": 4790, + "selected_both_wrong": 1627 + }, + { + "budget": 0.86, + "threshold": -1.5267945528030396, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.8109494693934922, + "rescued": 4625, + "harmed": 1062, + "missed_rescue": 1, + "avoided_harm": 71, + "selected_both_correct": 4922, + "selected_both_wrong": 1627 + }, + { + "budget": 0.87, + "threshold": -1.722639560699463, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.8106683533628505, + "rescued": 4625, + "harmed": 1066, + "missed_rescue": 1, + "avoided_harm": 67, + "selected_both_correct": 5061, + "selected_both_wrong": 1627 + }, + { + "budget": 0.88, + "threshold": -1.9391975402832031, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.8101061213015672, + "rescued": 4625, + "harmed": 1074, + "missed_rescue": 1, + "avoided_harm": 59, + "selected_both_correct": 5195, + "selected_both_wrong": 1627 + }, + { + "budget": 0.89, + "threshold": -2.1973628997802734, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.809895284278586, + "rescued": 4626, + "harmed": 1078, + "missed_rescue": 0, + "avoided_harm": 55, + "selected_both_correct": 5332, + "selected_both_wrong": 1627 + }, + { + "budget": 0.9, + "threshold": -2.4298460483551025, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.8094033312249631, + "rescued": 4626, + "harmed": 1085, + "missed_rescue": 0, + "avoided_harm": 48, + "selected_both_correct": 5468, + "selected_both_wrong": 1627 + }, + { + "budget": 0.91, + "threshold": -2.712864398956299, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.808700541148359, + "rescued": 4626, + "harmed": 1095, + "missed_rescue": 0, + "avoided_harm": 38, + "selected_both_correct": 5600, + "selected_both_wrong": 1627 + }, + { + "budget": 0.92, + "threshold": -2.9879889488220215, + "operator": "ge", + "actual_fraction": 0.9199522102747909, + "metric": 0.8082085880947361, + "rescued": 4626, + "harmed": 1102, + "missed_rescue": 0, + "avoided_harm": 31, + "selected_both_correct": 5735, + "selected_both_wrong": 1627 + }, + { + "budget": 0.93, + "threshold": -3.3283321857452393, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.8079274720640944, + "rescued": 4626, + "harmed": 1106, + "missed_rescue": 0, + "avoided_harm": 27, + "selected_both_correct": 5873, + "selected_both_wrong": 1627 + }, + { + "budget": 0.94, + "threshold": -3.705960750579834, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.8075057980181319, + "rescued": 4626, + "harmed": 1112, + "missed_rescue": 0, + "avoided_harm": 21, + "selected_both_correct": 6010, + "selected_both_wrong": 1627 + }, + { + "budget": 0.95, + "threshold": -4.111478805541992, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.8072246819874903, + "rescued": 4626, + "harmed": 1116, + "missed_rescue": 0, + "avoided_harm": 17, + "selected_both_correct": 6148, + "selected_both_wrong": 1627 + }, + { + "budget": 0.96, + "threshold": -4.588022232055664, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.8068030079415278, + "rescued": 4626, + "harmed": 1122, + "missed_rescue": 0, + "avoided_harm": 11, + "selected_both_correct": 6284, + "selected_both_wrong": 1627 + }, + { + "budget": 0.97, + "threshold": -5.219831466674805, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.8064516129032258, + "rescued": 4626, + "harmed": 1127, + "missed_rescue": 0, + "avoided_harm": 6, + "selected_both_correct": 6422, + "selected_both_wrong": 1627 + }, + { + "budget": 0.98, + "threshold": -6.1629791259765625, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.8062407758802446, + "rescued": 4626, + "harmed": 1130, + "missed_rescue": 0, + "avoided_harm": 3, + "selected_both_correct": 6561, + "selected_both_wrong": 1627 + }, + { + "budget": 0.99, + "threshold": -7.165260314941406, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.8061704968725841, + "rescued": 4626, + "harmed": 1131, + "missed_rescue": 0, + "avoided_harm": 2, + "selected_both_correct": 6702, + "selected_both_wrong": 1627 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8060299388572634, + "rescued": 4626, + "harmed": 1133, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 6843, + "selected_both_wrong": 1627 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.26688029946402747, + "rescued": 0, + "harmed": 0, + "missed_rescue": 18820, + "avoided_harm": 1970, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 21.90387725830078, + "operator": "ge", + "actual_fraction": 0.006891075631681933, + "metric": 0.2735161500723138, + "rescued": 234, + "harmed": 0, + "missed_rescue": 18586, + "avoided_harm": 1970, + "selected_both_correct": 0, + "selected_both_wrong": 9 + }, + { + "budget": 0.02, + "threshold": 20.418020248413086, + "operator": "ge", + "actual_fraction": 0.01417916796642373, + "metric": 0.28037886736806283, + "rescued": 476, + "harmed": 0, + "missed_rescue": 18344, + "avoided_harm": 1970, + "selected_both_correct": 0, + "selected_both_wrong": 24 + }, + { + "budget": 0.03, + "threshold": 19.33527946472168, + "operator": "ge", + "actual_fraction": 0.022970252105606444, + "metric": 0.2885460681167229, + "rescued": 764, + "harmed": 0, + "missed_rescue": 18056, + "avoided_harm": 1970, + "selected_both_correct": 0, + "selected_both_wrong": 46 + }, + { + "budget": 0.04, + "threshold": 18.620174407958984, + "operator": "ge", + "actual_fraction": 0.030173269432549697, + "metric": 0.2953237104046735, + "rescued": 1003, + "harmed": 0, + "missed_rescue": 17817, + "avoided_harm": 1970, + "selected_both_correct": 0, + "selected_both_wrong": 61 + }, + { + "budget": 0.05, + "threshold": 17.99974250793457, + "operator": "ge", + "actual_fraction": 0.036667328361171765, + "metric": 0.30156254430989987, + "rescued": 1223, + "harmed": 0, + "missed_rescue": 17597, + "avoided_harm": 1970, + "selected_both_correct": 0, + "selected_both_wrong": 70 + }, + { + "budget": 0.06, + "threshold": 17.459346771240234, + "operator": "ge", + "actual_fraction": 0.04265093724300258, + "metric": 0.30712077815273797, + "rescued": 1419, + "harmed": 0, + "missed_rescue": 17401, + "avoided_harm": 1970, + "selected_both_correct": 0, + "selected_both_wrong": 85 + }, + { + "budget": 0.07, + "threshold": 16.8415470123291, + "operator": "ge", + "actual_fraction": 0.05149873805405099, + "metric": 0.3152029038935995, + "rescued": 1705, + "harmed": 1, + "missed_rescue": 17115, + "avoided_harm": 1969, + "selected_both_correct": 0, + "selected_both_wrong": 110 + }, + { + "budget": 0.08, + "threshold": 16.37276268005371, + "operator": "ge", + "actual_fraction": 0.05833309701386723, + "metric": 0.3213283044550946, + "rescued": 1921, + "harmed": 1, + "missed_rescue": 16899, + "avoided_harm": 1969, + "selected_both_correct": 0, + "selected_both_wrong": 135 + }, + { + "budget": 0.09, + "threshold": 15.89695930480957, + "operator": "ge", + "actual_fraction": 0.06539432266114624, + "metric": 0.3276238550321867, + "rescued": 2143, + "harmed": 1, + "missed_rescue": 16677, + "avoided_harm": 1969, + "selected_both_correct": 0, + "selected_both_wrong": 162 + }, + { + "budget": 0.1, + "threshold": 15.354147911071777, + "operator": "ge", + "actual_fraction": 0.07466749851118737, + "metric": 0.33542239741371976, + "rescued": 2418, + "harmed": 1, + "missed_rescue": 16402, + "avoided_harm": 1969, + "selected_both_correct": 1, + "selected_both_wrong": 213 + }, + { + "budget": 0.11, + "threshold": 14.913481712341309, + "operator": "ge", + "actual_fraction": 0.08221081586932479, + "metric": 0.3420582480220061, + "rescued": 2653, + "harmed": 2, + "missed_rescue": 16167, + "avoided_harm": 1968, + "selected_both_correct": 1, + "selected_both_wrong": 243 + }, + { + "budget": 0.12, + "threshold": 14.473790168762207, + "operator": "ge", + "actual_fraction": 0.09009443325865638, + "metric": 0.3487791736380909, + "rescued": 2890, + "harmed": 2, + "missed_rescue": 15930, + "avoided_harm": 1968, + "selected_both_correct": 1, + "selected_both_wrong": 284 + }, + { + "budget": 0.13, + "threshold": 14.001596450805664, + "operator": "ge", + "actual_fraction": 0.09970790913989167, + "metric": 0.3563508493321612, + "rescued": 3159, + "harmed": 4, + "missed_rescue": 15661, + "avoided_harm": 1966, + "selected_both_correct": 2, + "selected_both_wrong": 351 + }, + { + "budget": 0.14, + "threshold": 13.631171226501465, + "operator": "ge", + "actual_fraction": 0.10767660153702181, + "metric": 0.3627598332529847, + "rescued": 3386, + "harmed": 5, + "missed_rescue": 15434, + "avoided_harm": 1965, + "selected_both_correct": 4, + "selected_both_wrong": 402 + }, + { + "budget": 0.15, + "threshold": 13.249896049499512, + "operator": "ge", + "actual_fraction": 0.11635425233247312, + "metric": 0.3697359838924652, + "rescued": 3632, + "harmed": 5, + "missed_rescue": 15188, + "avoided_harm": 1965, + "selected_both_correct": 4, + "selected_both_wrong": 462 + }, + { + "budget": 0.16, + "threshold": 12.844018936157227, + "operator": "ge", + "actual_fraction": 0.12576921986217848, + "metric": 0.3771942262428041, + "rescued": 3896, + "harmed": 6, + "missed_rescue": 14924, + "avoided_harm": 1964, + "selected_both_correct": 4, + "selected_both_wrong": 529 + }, + { + "budget": 0.17, + "threshold": 12.422953605651855, + "operator": "ge", + "actual_fraction": 0.13589314579020503, + "metric": 0.38487933528060575, + "rescued": 4169, + "harmed": 8, + "missed_rescue": 14651, + "avoided_harm": 1962, + "selected_both_correct": 4, + "selected_both_wrong": 611 + }, + { + "budget": 0.18, + "threshold": 12.03639030456543, + "operator": "ge", + "actual_fraction": 0.1462722967416272, + "metric": 0.3932450443807957, + "rescued": 4465, + "harmed": 9, + "missed_rescue": 14355, + "avoided_harm": 1961, + "selected_both_correct": 4, + "selected_both_wrong": 680 + }, + { + "budget": 0.19, + "threshold": 11.710915565490723, + "operator": "ge", + "actual_fraction": 0.15483651419334712, + "metric": 0.3996823866375521, + "rescued": 4695, + "harmed": 12, + "missed_rescue": 14125, + "avoided_harm": 1958, + "selected_both_correct": 4, + "selected_both_wrong": 749 + }, + { + "budget": 0.2, + "threshold": 11.339479446411133, + "operator": "ge", + "actual_fraction": 0.16600969855088904, + "metric": 0.4085585457845334, + "rescued": 5008, + "harmed": 12, + "missed_rescue": 13812, + "avoided_harm": 1958, + "selected_both_correct": 5, + "selected_both_wrong": 829 + }, + { + "budget": 0.21, + "threshold": 11.019915580749512, + "operator": "ge", + "actual_fraction": 0.1751977993931316, + "metric": 0.4154779797521481, + "rescued": 5255, + "harmed": 15, + "missed_rescue": 13565, + "avoided_harm": 1955, + "selected_both_correct": 8, + "selected_both_wrong": 900 + }, + { + "budget": 0.22, + "threshold": 10.663690567016602, + "operator": "ge", + "actual_fraction": 0.18844114227377137, + "metric": 0.4248362306099878, + "rescued": 5593, + "harmed": 23, + "missed_rescue": 13227, + "avoided_harm": 1947, + "selected_both_correct": 10, + "selected_both_wrong": 1019 + }, + { + "budget": 0.23, + "threshold": 10.3711576461792, + "operator": "ge", + "actual_fraction": 0.19884865156112638, + "metric": 0.4324362646399909, + "rescued": 5867, + "harmed": 29, + "missed_rescue": 12953, + "avoided_harm": 1941, + "selected_both_correct": 13, + "selected_both_wrong": 1103 + }, + { + "budget": 0.24, + "threshold": 10.053378105163574, + "operator": "ge", + "actual_fraction": 0.21183676941837054, + "metric": 0.44202138218529335, + "rescued": 6216, + "harmed": 40, + "missed_rescue": 12604, + "avoided_harm": 1930, + "selected_both_correct": 18, + "selected_both_wrong": 1196 + }, + { + "budget": 0.25, + "threshold": 9.729846000671387, + "operator": "ge", + "actual_fraction": 0.22570399568953295, + "metric": 0.45228709979298415, + "rescued": 6588, + "harmed": 50, + "missed_rescue": 12232, + "avoided_harm": 1920, + "selected_both_correct": 26, + "selected_both_wrong": 1295 + }, + { + "budget": 0.26, + "threshold": 9.389237403869629, + "operator": "ge", + "actual_fraction": 0.2447324391004736, + "metric": 0.4654453676658254, + "rescued": 7072, + "harmed": 70, + "missed_rescue": 11748, + "avoided_harm": 1900, + "selected_both_correct": 34, + "selected_both_wrong": 1454 + }, + { + "budget": 0.27, + "threshold": 9.115911483764648, + "operator": "ge", + "actual_fraction": 0.2614071406289879, + "metric": 0.47701556872642714, + "rescued": 7499, + "harmed": 89, + "missed_rescue": 11321, + "avoided_harm": 1881, + "selected_both_correct": 47, + "selected_both_wrong": 1583 + }, + { + "budget": 0.28, + "threshold": 8.828272819519043, + "operator": "ge", + "actual_fraction": 0.27870572554802486, + "metric": 0.4883021864277004, + "rescued": 7923, + "harmed": 115, + "missed_rescue": 10897, + "avoided_harm": 1855, + "selected_both_correct": 64, + "selected_both_wrong": 1726 + }, + { + "budget": 0.29, + "threshold": 8.522764205932617, + "operator": "ge", + "actual_fraction": 0.29937895244307067, + "metric": 0.5028216544253183, + "rescued": 8449, + "harmed": 129, + "missed_rescue": 10371, + "avoided_harm": 1841, + "selected_both_correct": 86, + "selected_both_wrong": 1893 + }, + { + "budget": 0.3, + "threshold": 8.2249174118042, + "operator": "ge", + "actual_fraction": 0.3208462127442362, + "metric": 0.5167172390324136, + "rescued": 8966, + "harmed": 156, + "missed_rescue": 9854, + "avoided_harm": 1814, + "selected_both_correct": 111, + "selected_both_wrong": 2081 + }, + { + "budget": 0.31, + "threshold": 7.953249454498291, + "operator": "ge", + "actual_fraction": 0.34197317301420754, + "metric": 0.530782973655106, + "rescued": 9490, + "harmed": 184, + "missed_rescue": 9330, + "avoided_harm": 1786, + "selected_both_correct": 138, + "selected_both_wrong": 2247 + }, + { + "budget": 0.32, + "threshold": 7.666618347167969, + "operator": "ge", + "actual_fraction": 0.3672971670022403, + "metric": 0.5470890168164932, + "rescued": 10101, + "harmed": 220, + "missed_rescue": 8719, + "avoided_harm": 1750, + "selected_both_correct": 176, + "selected_both_wrong": 2455 + }, + { + "budget": 0.33, + "threshold": 7.3823370933532715, + "operator": "ge", + "actual_fraction": 0.39228086095907894, + "metric": 0.5623174432124323, + "rescued": 10686, + "harmed": 268, + "missed_rescue": 8134, + "avoided_harm": 1702, + "selected_both_correct": 215, + "selected_both_wrong": 2664 + }, + { + "budget": 0.34, + "threshold": 7.096353530883789, + "operator": "ge", + "actual_fraction": 0.4180302299861044, + "metric": 0.5775175112724386, + "rescued": 11279, + "harmed": 325, + "missed_rescue": 7541, + "avoided_harm": 1645, + "selected_both_correct": 256, + "selected_both_wrong": 2881 + }, + { + "budget": 0.35, + "threshold": 6.865462779998779, + "operator": "ge", + "actual_fraction": 0.4408303320761138, + "metric": 0.5909877208405411, + "rescued": 11807, + "harmed": 378, + "missed_rescue": 7013, + "avoided_harm": 1592, + "selected_both_correct": 317, + "selected_both_wrong": 3043 + }, + { + "budget": 0.36, + "threshold": 6.669365882873535, + "operator": "ge", + "actual_fraction": 0.46053937554944274, + "metric": 0.6018773218387545, + "rescued": 12250, + "harmed": 437, + "missed_rescue": 6570, + "avoided_harm": 1533, + "selected_both_correct": 372, + "selected_both_wrong": 3181 + }, + { + "budget": 0.37, + "threshold": 6.469961166381836, + "operator": "ge", + "actual_fraction": 0.4794827439525849, + "metric": 0.6114907977199898, + "rescued": 12638, + "harmed": 486, + "missed_rescue": 6182, + "avoided_harm": 1484, + "selected_both_correct": 419, + "selected_both_wrong": 3365 + }, + { + "budget": 0.38, + "threshold": 6.239957809448242, + "operator": "ge", + "actual_fraction": 0.5016873209880044, + "metric": 0.6235998071633156, + "rescued": 13124, + "harmed": 545, + "missed_rescue": 5696, + "avoided_harm": 1425, + "selected_both_correct": 470, + "selected_both_wrong": 3552 + }, + { + "budget": 0.39, + "threshold": 6.02949333190918, + "operator": "ge", + "actual_fraction": 0.5211978561098035, + "metric": 0.6340640331225363, + "rescued": 13540, + "harmed": 592, + "missed_rescue": 5280, + "avoided_harm": 1378, + "selected_both_correct": 532, + "selected_both_wrong": 3715 + }, + { + "budget": 0.4, + "threshold": 5.813343048095703, + "operator": "ge", + "actual_fraction": 0.5422397413719763, + "metric": 0.6448402007770184, + "rescued": 13973, + "harmed": 645, + "missed_rescue": 4847, + "avoided_harm": 1325, + "selected_both_correct": 606, + "selected_both_wrong": 3897 + }, + { + "budget": 0.41, + "threshold": 5.622989177703857, + "operator": "ge", + "actual_fraction": 0.5594532512832147, + "metric": 0.6537163599239997, + "rescued": 14328, + "harmed": 687, + "missed_rescue": 4492, + "avoided_harm": 1283, + "selected_both_correct": 659, + "selected_both_wrong": 4054 + }, + { + "budget": 0.42, + "threshold": 5.466009616851807, + "operator": "ge", + "actual_fraction": 0.5745682443354224, + "metric": 0.6610044522587415, + "rescued": 14636, + "harmed": 738, + "missed_rescue": 4184, + "avoided_harm": 1232, + "selected_both_correct": 725, + "selected_both_wrong": 4162 + }, + { + "budget": 0.43, + "threshold": 5.273525238037109, + "operator": "ge", + "actual_fraction": 0.593228029379236, + "metric": 0.6689447863199387, + "rescued": 14973, + "harmed": 795, + "missed_rescue": 3847, + "avoided_harm": 1175, + "selected_both_correct": 811, + "selected_both_wrong": 4340 + }, + { + "budget": 0.44, + "threshold": 5.097577095031738, + "operator": "ge", + "actual_fraction": 0.607350480673794, + "metric": 0.675041828545501, + "rescued": 15230, + "harmed": 837, + "missed_rescue": 3590, + "avoided_harm": 1133, + "selected_both_correct": 865, + "selected_both_wrong": 4485 + }, + { + "budget": 0.45, + "threshold": 4.9260430335998535, + "operator": "ge", + "actual_fraction": 0.6220968153588747, + "metric": 0.6821314125287128, + "rescued": 15514, + "harmed": 871, + "missed_rescue": 3306, + "avoided_harm": 1099, + "selected_both_correct": 940, + "selected_both_wrong": 4612 + }, + { + "budget": 0.46, + "threshold": 4.770597457885742, + "operator": "ge", + "actual_fraction": 0.6337237330913422, + "metric": 0.6876612880356181, + "rescued": 15744, + "harmed": 906, + "missed_rescue": 3076, + "avoided_harm": 1064, + "selected_both_correct": 986, + "selected_both_wrong": 4711 + }, + { + "budget": 0.47, + "threshold": 4.615985870361328, + "operator": "ge", + "actual_fraction": 0.6452939341519439, + "metric": 0.6922269801208065, + "rescued": 15939, + "harmed": 940, + "missed_rescue": 2881, + "avoided_harm": 1030, + "selected_both_correct": 1057, + "selected_both_wrong": 4819 + }, + { + "budget": 0.48, + "threshold": 4.460105895996094, + "operator": "ge", + "actual_fraction": 0.6573178685874713, + "metric": 0.6969344638856592, + "rescued": 16133, + "harmed": 968, + "missed_rescue": 2687, + "avoided_harm": 1002, + "selected_both_correct": 1133, + "selected_both_wrong": 4945 + }, + { + "budget": 0.49, + "threshold": 4.310606479644775, + "operator": "ge", + "actual_fraction": 0.6692283696792672, + "metric": 0.7020389643535717, + "rescued": 16345, + "harmed": 1000, + "missed_rescue": 2475, + "avoided_harm": 970, + "selected_both_correct": 1194, + "selected_both_wrong": 5060 + }, + { + "budget": 0.5, + "threshold": 4.1707234382629395, + "operator": "ge", + "actual_fraction": 0.6794090122791595, + "metric": 0.7056688313529762, + "rescued": 16510, + "harmed": 1037, + "missed_rescue": 2310, + "avoided_harm": 933, + "selected_both_correct": 1255, + "selected_both_wrong": 5156 + }, + { + "budget": 0.51, + "threshold": 4.034754753112793, + "operator": "ge", + "actual_fraction": 0.6898448799024474, + "metric": 0.7097240733913734, + "rescued": 16687, + "harmed": 1071, + "missed_rescue": 2133, + "avoided_harm": 899, + "selected_both_correct": 1314, + "selected_both_wrong": 5254 + }, + { + "budget": 0.52, + "threshold": 3.9028453826904297, + "operator": "ge", + "actual_fraction": 0.6984658140260329, + "metric": 0.7130136403595837, + "rescued": 16833, + "harmed": 1101, + "missed_rescue": 1987, + "avoided_harm": 869, + "selected_both_correct": 1371, + "selected_both_wrong": 5325 + }, + { + "budget": 0.53, + "threshold": 3.742058277130127, + "operator": "ge", + "actual_fraction": 0.7087031732977909, + "metric": 0.7167569407027196, + "rescued": 16998, + "harmed": 1134, + "missed_rescue": 1822, + "avoided_harm": 836, + "selected_both_correct": 1441, + "selected_both_wrong": 5418 + }, + { + "budget": 0.54, + "threshold": 3.5907771587371826, + "operator": "ge", + "actual_fraction": 0.7182032158352948, + "metric": 0.7202450160224598, + "rescued": 17146, + "harmed": 1159, + "missed_rescue": 1674, + "avoided_harm": 811, + "selected_both_correct": 1518, + "selected_both_wrong": 5503 + }, + { + "budget": 0.55, + "threshold": 3.454869270324707, + "operator": "ge", + "actual_fraction": 0.7268808666307461, + "metric": 0.7235912996625358, + "rescued": 17292, + "harmed": 1187, + "missed_rescue": 1528, + "avoided_harm": 783, + "selected_both_correct": 1594, + "selected_both_wrong": 5559 + }, + { + "budget": 0.56, + "threshold": 3.3179640769958496, + "operator": "ge", + "actual_fraction": 0.7349629923716077, + "metric": 0.7256898165215665, + "rescued": 17400, + "harmed": 1221, + "missed_rescue": 1420, + "avoided_harm": 749, + "selected_both_correct": 1665, + "selected_both_wrong": 5631 + }, + { + "budget": 0.57, + "threshold": 3.192162036895752, + "operator": "ge", + "actual_fraction": 0.7420242180188866, + "metric": 0.7278734083883958, + "rescued": 17501, + "harmed": 1245, + "missed_rescue": 1319, + "avoided_harm": 725, + "selected_both_correct": 1739, + "selected_both_wrong": 5681 + }, + { + "budget": 0.58, + "threshold": 3.0637471675872803, + "operator": "ge", + "actual_fraction": 0.7489152936505686, + "metric": 0.729886850239628, + "rescued": 17599, + "harmed": 1272, + "missed_rescue": 1221, + "avoided_harm": 698, + "selected_both_correct": 1800, + "selected_both_wrong": 5738 + }, + { + "budget": 0.59, + "threshold": 2.9273691177368164, + "operator": "ge", + "actual_fraction": 0.7562033859853103, + "metric": 0.731361483708136, + "rescued": 17685, + "harmed": 1306, + "missed_rescue": 1135, + "avoided_harm": 664, + "selected_both_correct": 1873, + "selected_both_wrong": 5802 + }, + { + "budget": 0.6, + "threshold": 2.7924067974090576, + "operator": "ge", + "actual_fraction": 0.7631228199529252, + "metric": 0.7332898505515696, + "rescued": 17771, + "harmed": 1324, + "missed_rescue": 1049, + "avoided_harm": 646, + "selected_both_correct": 1946, + "selected_both_wrong": 5869 + }, + { + "budget": 0.61, + "threshold": 2.633131504058838, + "operator": "ge", + "actual_fraction": 0.7713750957093838, + "metric": 0.7350197090434734, + "rescued": 17859, + "harmed": 1351, + "missed_rescue": 961, + "avoided_harm": 619, + "selected_both_correct": 2052, + "selected_both_wrong": 5939 + }, + { + "budget": 0.62, + "threshold": 2.5002918243408203, + "operator": "ge", + "actual_fraction": 0.7771318379037518, + "metric": 0.7361540424807872, + "rescued": 17916, + "harmed": 1368, + "missed_rescue": 904, + "avoided_harm": 602, + "selected_both_correct": 2131, + "selected_both_wrong": 5989 + }, + { + "budget": 0.63, + "threshold": 2.3700814247131348, + "operator": "ge", + "actual_fraction": 0.7839378385276352, + "metric": 0.7381107676601537, + "rescued": 18000, + "harmed": 1383, + "missed_rescue": 820, + "avoided_harm": 587, + "selected_both_correct": 2215, + "selected_both_wrong": 6046 + }, + { + "budget": 0.64, + "threshold": 2.223950147628784, + "operator": "ge", + "actual_fraction": 0.790488614128123, + "metric": 0.7389615177381391, + "rescued": 18056, + "harmed": 1409, + "missed_rescue": 764, + "avoided_harm": 561, + "selected_both_correct": 2311, + "selected_both_wrong": 6099 + }, + { + "budget": 0.65, + "threshold": 2.0700783729553223, + "operator": "ge", + "actual_fraction": 0.7976349147832005, + "metric": 0.7402092845191844, + "rescued": 18121, + "harmed": 1430, + "missed_rescue": 699, + "avoided_harm": 540, + "selected_both_correct": 2411, + "selected_both_wrong": 6165 + }, + { + "budget": 0.66, + "threshold": 1.9317810535430908, + "operator": "ge", + "actual_fraction": 0.8047812154382781, + "metric": 0.741768992995491, + "rescued": 18193, + "harmed": 1447, + "missed_rescue": 627, + "avoided_harm": 523, + "selected_both_correct": 2516, + "selected_both_wrong": 6223 + }, + { + "budget": 0.67, + "threshold": 1.7615712881088257, + "operator": "ge", + "actual_fraction": 0.8127215494994754, + "metric": 0.7430167597765364, + "rescued": 18260, + "harmed": 1470, + "missed_rescue": 560, + "avoided_harm": 500, + "selected_both_correct": 2650, + "selected_both_wrong": 6279 + }, + { + "budget": 0.68, + "threshold": 1.6136956214904785, + "operator": "ge", + "actual_fraction": 0.819669341803023, + "metric": 0.7437824348467232, + "rescued": 18307, + "harmed": 1490, + "missed_rescue": 513, + "avoided_harm": 480, + "selected_both_correct": 2778, + "selected_both_wrong": 6329 + }, + { + "budget": 0.69, + "threshold": 1.4685550928115845, + "operator": "ge", + "actual_fraction": 0.826532059098772, + "metric": 0.7450018432918356, + "rescued": 18364, + "harmed": 1504, + "missed_rescue": 456, + "avoided_harm": 466, + "selected_both_correct": 2907, + "selected_both_wrong": 6371 + }, + { + "budget": 0.7, + "threshold": 1.327118992805481, + "operator": "ge", + "actual_fraction": 0.8331395513711255, + "metric": 0.7465048350962765, + "rescued": 18428, + "harmed": 1515, + "missed_rescue": 392, + "avoided_harm": 455, + "selected_both_correct": 3018, + "selected_both_wrong": 6418 + }, + { + "budget": 0.71, + "threshold": 1.1890794038772583, + "operator": "ge", + "actual_fraction": 0.8396052519638147, + "metric": 0.7474406601820606, + "rescued": 18479, + "harmed": 1533, + "missed_rescue": 341, + "avoided_harm": 437, + "selected_both_correct": 3134, + "selected_both_wrong": 6461 + }, + { + "budget": 0.72, + "threshold": 1.042020320892334, + "operator": "ge", + "actual_fraction": 0.8465246859314296, + "metric": 0.7486884269631058, + "rescued": 18534, + "harmed": 1544, + "missed_rescue": 286, + "avoided_harm": 426, + "selected_both_correct": 3258, + "selected_both_wrong": 6515 + }, + { + "budget": 0.73, + "threshold": 0.8947708606719971, + "operator": "ge", + "actual_fraction": 0.853869494938037, + "metric": 0.7494541020332927, + "rescued": 18575, + "harmed": 1558, + "missed_rescue": 245, + "avoided_harm": 412, + "selected_both_correct": 3397, + "selected_both_wrong": 6580 + }, + { + "budget": 0.74, + "threshold": 0.7377536296844482, + "operator": "ge", + "actual_fraction": 0.8616396789836372, + "metric": 0.7501630604316138, + "rescued": 18618, + "harmed": 1576, + "missed_rescue": 202, + "avoided_harm": 394, + "selected_both_correct": 3554, + "selected_both_wrong": 6636 + }, + { + "budget": 0.75, + "threshold": 0.5805552005767822, + "operator": "ge", + "actual_fraction": 0.8682755295919236, + "metric": 0.7506451521424723, + "rescued": 18651, + "harmed": 1592, + "missed_rescue": 169, + "avoided_harm": 378, + "selected_both_correct": 3684, + "selected_both_wrong": 6691 + }, + { + "budget": 0.76, + "threshold": 0.41288113594055176, + "operator": "ge", + "actual_fraction": 0.8752233218954711, + "metric": 0.7511272438533307, + "rescued": 18683, + "harmed": 1607, + "missed_rescue": 137, + "avoided_harm": 363, + "selected_both_correct": 3826, + "selected_both_wrong": 6747 + }, + { + "budget": 0.77, + "threshold": 0.23000085353851318, + "operator": "ge", + "actual_fraction": 0.8818875308396903, + "metric": 0.7516093355641891, + "rescued": 18710, + "harmed": 1617, + "missed_rescue": 110, + "avoided_harm": 353, + "selected_both_correct": 3989, + "selected_both_wrong": 6782 + }, + { + "budget": 0.78, + "threshold": 0.06256341934204102, + "operator": "ge", + "actual_fraction": 0.8885233814479766, + "metric": 0.7519496355953833, + "rescued": 18738, + "harmed": 1633, + "missed_rescue": 82, + "avoided_harm": 337, + "selected_both_correct": 4148, + "selected_both_wrong": 6813 + }, + { + "budget": 0.79, + "threshold": -0.12610122561454773, + "operator": "ge", + "actual_fraction": 0.8961234154779798, + "metric": 0.7518645605875848, + "rescued": 18753, + "harmed": 1651, + "missed_rescue": 67, + "avoided_harm": 319, + "selected_both_correct": 4348, + "selected_both_wrong": 6848 + }, + { + "budget": 0.8, + "threshold": -0.3212296962738037, + "operator": "ge", + "actual_fraction": 0.9025040410628704, + "metric": 0.7519779939313161, + "rescued": 18767, + "harmed": 1661, + "missed_rescue": 53, + "avoided_harm": 309, + "selected_both_correct": 4518, + "selected_both_wrong": 6879 + }, + { + "budget": 0.81, + "threshold": -0.533017635345459, + "operator": "ge", + "actual_fraction": 0.9087145166321641, + "metric": 0.7517794855797861, + "rescued": 18774, + "harmed": 1675, + "missed_rescue": 46, + "avoided_harm": 295, + "selected_both_correct": 4691, + "selected_both_wrong": 6904 + }, + { + "budget": 0.82, + "threshold": -0.715209424495697, + "operator": "ge", + "actual_fraction": 0.9148115588577262, + "metric": 0.7517227689079204, + "rescued": 18784, + "harmed": 1687, + "missed_rescue": 36, + "avoided_harm": 283, + "selected_both_correct": 4873, + "selected_both_wrong": 6915 + }, + { + "budget": 0.83, + "threshold": -0.9083468914031982, + "operator": "ge", + "actual_fraction": 0.9199160593256388, + "metric": 0.7516376939001219, + "rescued": 18790, + "harmed": 1696, + "missed_rescue": 30, + "avoided_harm": 274, + "selected_both_correct": 5027, + "selected_both_wrong": 6926 + }, + { + "budget": 0.84, + "threshold": -1.103330135345459, + "operator": "ge", + "actual_fraction": 0.9256444431840739, + "metric": 0.7513824688767263, + "rescued": 18799, + "harmed": 1714, + "missed_rescue": 21, + "avoided_harm": 256, + "selected_both_correct": 5188, + "selected_both_wrong": 6940 + }, + { + "budget": 0.85, + "threshold": -1.338437557220459, + "operator": "ge", + "actual_fraction": 0.9322802937923602, + "metric": 0.7511272438533307, + "rescued": 18803, + "harmed": 1727, + "missed_rescue": 17, + "avoided_harm": 243, + "selected_both_correct": 5388, + "selected_both_wrong": 6957 + }, + { + "budget": 0.86, + "threshold": -1.5267945528030396, + "operator": "ge", + "actual_fraction": 0.9366474775260187, + "metric": 0.7509287355018007, + "rescued": 18808, + "harmed": 1739, + "missed_rescue": 12, + "avoided_harm": 231, + "selected_both_correct": 5519, + "selected_both_wrong": 6963 + }, + { + "budget": 0.87, + "threshold": -1.722639560699463, + "operator": "ge", + "actual_fraction": 0.9409579445878116, + "metric": 0.7506167938065395, + "rescued": 18812, + "harmed": 1754, + "missed_rescue": 8, + "avoided_harm": 216, + "selected_both_correct": 5647, + "selected_both_wrong": 6968 + }, + { + "budget": 0.88, + "threshold": -1.9391975402832031, + "operator": "ge", + "actual_fraction": 0.9455519950089328, + "metric": 0.7501630604316138, + "rescued": 18812, + "harmed": 1770, + "missed_rescue": 8, + "avoided_harm": 200, + "selected_both_correct": 5790, + "selected_both_wrong": 6971 + }, + { + "budget": 0.89, + "threshold": -2.1973628997802734, + "operator": "ge", + "actual_fraction": 0.9506281371409125, + "metric": 0.7498227604004197, + "rescued": 18812, + "harmed": 1782, + "missed_rescue": 8, + "avoided_harm": 188, + "selected_both_correct": 5941, + "selected_both_wrong": 6987 + }, + { + "budget": 0.9, + "threshold": -2.4298460483551025, + "operator": "ge", + "actual_fraction": 0.954825170858974, + "metric": 0.7494541020332927, + "rescued": 18812, + "harmed": 1795, + "missed_rescue": 8, + "avoided_harm": 175, + "selected_both_correct": 6070, + "selected_both_wrong": 6993 + }, + { + "budget": 0.91, + "threshold": -2.712864398956299, + "operator": "ge", + "actual_fraction": 0.9591639962566997, + "metric": 0.7490287269943, + "rescued": 18812, + "harmed": 1810, + "missed_rescue": 8, + "avoided_harm": 160, + "selected_both_correct": 6203, + "selected_both_wrong": 6998 + }, + { + "budget": 0.92, + "threshold": -2.9879889488220215, + "operator": "ge", + "actual_fraction": 0.9639565550293508, + "metric": 0.7488018603068372, + "rescued": 18814, + "harmed": 1820, + "missed_rescue": 6, + "avoided_harm": 150, + "selected_both_correct": 6350, + "selected_both_wrong": 7008 + }, + { + "budget": 0.93, + "threshold": -3.3283321857452393, + "operator": "ge", + "actual_fraction": 0.9685789637864051, + "metric": 0.7485182769475087, + "rescued": 18814, + "harmed": 1830, + "missed_rescue": 6, + "avoided_harm": 140, + "selected_both_correct": 6500, + "selected_both_wrong": 7011 + }, + { + "budget": 0.94, + "threshold": -3.705960750579834, + "operator": "ge", + "actual_fraction": 0.9733715225590562, + "metric": 0.7482346935881803, + "rescued": 18814, + "harmed": 1840, + "missed_rescue": 6, + "avoided_harm": 130, + "selected_both_correct": 6653, + "selected_both_wrong": 7017 + }, + { + "budget": 0.95, + "threshold": -4.111478805541992, + "operator": "ge", + "actual_fraction": 0.978050647987976, + "metric": 0.7477526018773218, + "rescued": 18815, + "harmed": 1858, + "missed_rescue": 5, + "avoided_harm": 112, + "selected_both_correct": 6793, + "selected_both_wrong": 7023 + }, + { + "budget": 0.96, + "threshold": -4.588022232055664, + "operator": "ge", + "actual_fraction": 0.981765589995179, + "metric": 0.7471003601508663, + "rescued": 18817, + "harmed": 1883, + "missed_rescue": 3, + "avoided_harm": 87, + "selected_both_correct": 6895, + "selected_both_wrong": 7025 + }, + { + "budget": 0.97, + "threshold": -5.219831466674805, + "operator": "ge", + "actual_fraction": 0.986614865439696, + "metric": 0.7466182684400079, + "rescued": 18817, + "harmed": 1900, + "missed_rescue": 3, + "avoided_harm": 70, + "selected_both_correct": 7047, + "selected_both_wrong": 7027 + }, + { + "budget": 0.98, + "threshold": -6.1629791259765625, + "operator": "ge", + "actual_fraction": 0.9918895159232056, + "metric": 0.7462779684088138, + "rescued": 18819, + "harmed": 1914, + "missed_rescue": 1, + "avoided_harm": 56, + "selected_both_correct": 7213, + "selected_both_wrong": 7031 + }, + { + "budget": 0.99, + "threshold": -7.165260314941406, + "operator": "ge", + "actual_fraction": 0.9958313246178714, + "metric": 0.7456257266823583, + "rescued": 18820, + "harmed": 1938, + "missed_rescue": 0, + "avoided_harm": 32, + "selected_both_correct": 7327, + "selected_both_wrong": 7031 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.7447182599325072, + "rescued": 18820, + "harmed": 1970, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 7441, + "selected_both_wrong": 7032 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 6843, + "rescuable": 4626, + "harmful": 1133, + "both_wrong": 1627, + "unit": "sample", + "total": 14229 + }, + "test": { + "both_correct": 7441, + "rescuable": 18820, + "harmful": 1970, + "both_wrong": 7032, + "unit": "sample", + "total": 35263 + } + }, + "small": { + "validation": 0.5605453650994447, + "test": 0.26688029946402747 + }, + "large": { + "validation": 0.8060299388572634, + "test": 0.7447182599325072 + }, + "learned_minus_confidence_test": -0.07770184045600204, + "learned_minus_small_test": 0.394124152794714, + "learned_minus_large_test": -0.08371380767376568 + }, + { + "key": [ + "grefcoco", + "instancevg", + "MiniCPM" + ], + "expert_display": "InstanceVG", + "run": "shared_12_instancevg_MiniCPM", + "validation_rows": 14229, + "test_rows": 35263, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detections_above_07", + "max_score", + "mean_score", + "query_count", + "detections_above_07_valid", + "max_score_valid", + "mean_score_valid", + "query_count_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/instancevg/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 65, + "validation_area_0_25": 0.7465176751704266, + "validation_point": { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.7716635041113219, + "rescued": 1117, + "harmed": 813, + "missed_rescue": 252, + "avoided_harm": 2108, + "selected_both_correct": 5984, + "selected_both_wrong": 1334 + }, + "test_point": { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.5209426310864078, + "metric": 0.6187788900547316, + "rescued": 1563, + "harmed": 3352, + "missed_rescue": 1064, + "avoided_harm": 5250, + "selected_both_correct": 8439, + "selected_both_wrong": 5016 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7502986857825568, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1369, + "avoided_harm": 2921, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -2.0005727208172175e-07, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.7498770117365943, + "rescued": 0, + "harmed": 6, + "missed_rescue": 1369, + "avoided_harm": 2915, + "selected_both_correct": 136, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -2.4156264544217265e-07, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.7497364537212735, + "rescued": 0, + "harmed": 8, + "missed_rescue": 1369, + "avoided_harm": 2913, + "selected_both_correct": 276, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -2.7729177531909954e-07, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.7494553376906318, + "rescued": 0, + "harmed": 12, + "missed_rescue": 1369, + "avoided_harm": 2909, + "selected_both_correct": 414, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": -3.064858447032748e-07, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.749314779675311, + "rescued": 0, + "harmed": 14, + "missed_rescue": 1369, + "avoided_harm": 2907, + "selected_both_correct": 555, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": -3.3848826319626824e-07, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.7492445006676506, + "rescued": 0, + "harmed": 15, + "missed_rescue": 1369, + "avoided_harm": 2906, + "selected_both_correct": 696, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": -3.668622525765386e-07, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.7491742216599901, + "rescued": 0, + "harmed": 16, + "missed_rescue": 1369, + "avoided_harm": 2905, + "selected_both_correct": 837, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": -3.98026287484754e-07, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.7491039426523297, + "rescued": 0, + "harmed": 17, + "missed_rescue": 1369, + "avoided_harm": 2904, + "selected_both_correct": 979, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": -4.302527258914779e-07, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.7488931056293485, + "rescued": 0, + "harmed": 20, + "missed_rescue": 1369, + "avoided_harm": 2901, + "selected_both_correct": 1118, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": -4.7191912244670675e-07, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.7485417105910465, + "rescued": 0, + "harmed": 25, + "missed_rescue": 1369, + "avoided_harm": 2896, + "selected_both_correct": 1255, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": -5.115426802149159e-07, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.748120036545084, + "rescued": 0, + "harmed": 31, + "missed_rescue": 1369, + "avoided_harm": 2890, + "selected_both_correct": 1391, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": -5.534793672268279e-07, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.7477686415067819, + "rescued": 0, + "harmed": 36, + "missed_rescue": 1369, + "avoided_harm": 2885, + "selected_both_correct": 1529, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": -5.941800509390305e-07, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.7472766884531591, + "rescued": 0, + "harmed": 43, + "missed_rescue": 1369, + "avoided_harm": 2878, + "selected_both_correct": 1664, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": -6.464504735959054e-07, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.746573898376555, + "rescued": 0, + "harmed": 53, + "missed_rescue": 1369, + "avoided_harm": 2868, + "selected_both_correct": 1796, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": -6.999837864896108e-07, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.7463630613535737, + "rescued": 0, + "harmed": 56, + "missed_rescue": 1369, + "avoided_harm": 2865, + "selected_both_correct": 1936, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": -7.663418841730163e-07, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.7461522243305925, + "rescued": 0, + "harmed": 59, + "missed_rescue": 1369, + "avoided_harm": 2862, + "selected_both_correct": 2075, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": -8.471382670904859e-07, + "operator": "ge", + "actual_fraction": 0.15995502143509735, + "metric": 0.74573055028463, + "rescued": 0, + "harmed": 65, + "missed_rescue": 1369, + "avoided_harm": 2856, + "selected_both_correct": 2211, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": -9.292756431023008e-07, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.7454494342539884, + "rescued": 0, + "harmed": 69, + "missed_rescue": 1369, + "avoided_harm": 2852, + "selected_both_correct": 2349, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": -1.0245233852401725e-06, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.7451683182233467, + "rescued": 0, + "harmed": 73, + "missed_rescue": 1369, + "avoided_harm": 2848, + "selected_both_correct": 2488, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": -1.1468601996966754e-06, + "operator": "ge", + "actual_fraction": 0.18996415770609318, + "metric": 0.7444655281467426, + "rescued": 0, + "harmed": 83, + "missed_rescue": 1369, + "avoided_harm": 2838, + "selected_both_correct": 2620, + "selected_both_wrong": 0 + }, + { + "budget": 0.2, + "threshold": -1.3018121762797819e-06, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.7439032960854592, + "rescued": 0, + "harmed": 91, + "missed_rescue": 1369, + "avoided_harm": 2830, + "selected_both_correct": 2754, + "selected_both_wrong": 0 + }, + { + "budget": 0.21, + "threshold": -1.4898060953782988e-06, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.7430599479935344, + "rescued": 0, + "harmed": 103, + "missed_rescue": 1369, + "avoided_harm": 2818, + "selected_both_correct": 2885, + "selected_both_wrong": 0 + }, + { + "budget": 0.22, + "threshold": -1.735574414851726e-06, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.7422868789092698, + "rescued": 0, + "harmed": 114, + "missed_rescue": 1369, + "avoided_harm": 2807, + "selected_both_correct": 3016, + "selected_both_wrong": 0 + }, + { + "budget": 0.23, + "threshold": -2.0881013824691763e-06, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.7413732518096845, + "rescued": 0, + "harmed": 127, + "missed_rescue": 1369, + "avoided_harm": 2794, + "selected_both_correct": 3144, + "selected_both_wrong": 1 + }, + { + "budget": 0.24, + "threshold": -2.5644872039265465e-06, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.7406704617330804, + "rescued": 0, + "harmed": 137, + "missed_rescue": 1369, + "avoided_harm": 2784, + "selected_both_correct": 3276, + "selected_both_wrong": 1 + }, + { + "budget": 0.25, + "threshold": -3.2399550491390983e-06, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.7400379506641366, + "rescued": 0, + "harmed": 146, + "missed_rescue": 1369, + "avoided_harm": 2775, + "selected_both_correct": 3410, + "selected_both_wrong": 1 + }, + { + "budget": 0.26, + "threshold": -4.2517635847616475e-06, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.7391946025722117, + "rescued": 0, + "harmed": 158, + "missed_rescue": 1369, + "avoided_harm": 2763, + "selected_both_correct": 3540, + "selected_both_wrong": 1 + }, + { + "budget": 0.27, + "threshold": -5.607922958006384e-06, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.738210696464966, + "rescued": 0, + "harmed": 172, + "missed_rescue": 1369, + "avoided_harm": 2749, + "selected_both_correct": 3668, + "selected_both_wrong": 1 + }, + { + "budget": 0.28, + "threshold": -8.438555596512742e-06, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.7374376273807014, + "rescued": 0, + "harmed": 183, + "missed_rescue": 1369, + "avoided_harm": 2738, + "selected_both_correct": 3799, + "selected_both_wrong": 2 + }, + { + "budget": 0.29, + "threshold": -1.3145449884177651e-05, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.7362428842504743, + "rescued": 0, + "harmed": 200, + "missed_rescue": 1369, + "avoided_harm": 2721, + "selected_both_correct": 3924, + "selected_both_wrong": 2 + }, + { + "budget": 0.3, + "threshold": -2.076632881653495e-05, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.7352589781432286, + "rescued": 1, + "harmed": 215, + "missed_rescue": 1368, + "avoided_harm": 2706, + "selected_both_correct": 4050, + "selected_both_wrong": 2 + }, + { + "budget": 0.31, + "threshold": -4.0114435250870883e-05, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.7342047930283224, + "rescued": 1, + "harmed": 230, + "missed_rescue": 1368, + "avoided_harm": 2691, + "selected_both_correct": 4177, + "selected_both_wrong": 2 + }, + { + "budget": 0.32, + "threshold": -7.415730942739174e-05, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.7327289338674537, + "rescued": 1, + "harmed": 251, + "missed_rescue": 1368, + "avoided_harm": 2670, + "selected_both_correct": 4299, + "selected_both_wrong": 2 + }, + { + "budget": 0.33, + "threshold": -0.00015223574882838875, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.7313233537142455, + "rescued": 1, + "harmed": 271, + "missed_rescue": 1368, + "avoided_harm": 2650, + "selected_both_correct": 4421, + "selected_both_wrong": 2 + }, + { + "budget": 0.34, + "threshold": -0.00032258269493468106, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.7298474945533769, + "rescued": 1, + "harmed": 292, + "missed_rescue": 1368, + "avoided_harm": 2629, + "selected_both_correct": 4541, + "selected_both_wrong": 3 + }, + { + "budget": 0.35, + "threshold": -0.0006007538759149611, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.7284419144001687, + "rescued": 1, + "harmed": 312, + "missed_rescue": 1368, + "avoided_harm": 2609, + "selected_both_correct": 4664, + "selected_both_wrong": 3 + }, + { + "budget": 0.36, + "threshold": -0.0011453087208792567, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.7268957762316396, + "rescued": 1, + "harmed": 334, + "missed_rescue": 1368, + "avoided_harm": 2587, + "selected_both_correct": 4784, + "selected_both_wrong": 3 + }, + { + "budget": 0.37, + "threshold": -0.002608026610687375, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.7252793590554502, + "rescued": 1, + "harmed": 357, + "missed_rescue": 1368, + "avoided_harm": 2564, + "selected_both_correct": 4903, + "selected_both_wrong": 3 + }, + { + "budget": 0.38, + "threshold": -0.006219128146767616, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.7234521048562794, + "rescued": 1, + "harmed": 383, + "missed_rescue": 1368, + "avoided_harm": 2538, + "selected_both_correct": 5019, + "selected_both_wrong": 4 + }, + { + "budget": 0.39, + "threshold": -0.01280172262340784, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.7216248506571087, + "rescued": 1, + "harmed": 409, + "missed_rescue": 1368, + "avoided_harm": 2512, + "selected_both_correct": 5134, + "selected_both_wrong": 5 + }, + { + "budget": 0.4, + "threshold": -0.03794781118631363, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.719446201419636, + "rescued": 1, + "harmed": 440, + "missed_rescue": 1368, + "avoided_harm": 2481, + "selected_both_correct": 5242, + "selected_both_wrong": 8 + }, + { + "budget": 0.41, + "threshold": -0.08479184657335281, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.7169161571438611, + "rescued": 1, + "harmed": 476, + "missed_rescue": 1368, + "avoided_harm": 2445, + "selected_both_correct": 5347, + "selected_both_wrong": 9 + }, + { + "budget": 0.42, + "threshold": -0.14941807091236115, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.7148077869140488, + "rescued": 3, + "harmed": 508, + "missed_rescue": 1366, + "avoided_harm": 2413, + "selected_both_correct": 5450, + "selected_both_wrong": 15 + }, + { + "budget": 0.43, + "threshold": -0.2939925789833069, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.7126994166842364, + "rescued": 5, + "harmed": 540, + "missed_rescue": 1364, + "avoided_harm": 2381, + "selected_both_correct": 5552, + "selected_both_wrong": 21 + }, + { + "budget": 0.44, + "threshold": -0.4386065602302551, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.7100288143931408, + "rescued": 5, + "harmed": 578, + "missed_rescue": 1364, + "avoided_harm": 2343, + "selected_both_correct": 5648, + "selected_both_wrong": 29 + }, + { + "budget": 0.45, + "threshold": -0.5420411825180054, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.7070770960714035, + "rescued": 5, + "harmed": 620, + "missed_rescue": 1364, + "avoided_harm": 2301, + "selected_both_correct": 5742, + "selected_both_wrong": 36 + }, + { + "budget": 0.46, + "threshold": -0.6230618953704834, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.7045470517956286, + "rescued": 6, + "harmed": 657, + "missed_rescue": 1363, + "avoided_harm": 2264, + "selected_both_correct": 5841, + "selected_both_wrong": 41 + }, + { + "budget": 0.47, + "threshold": -0.6914289593696594, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.7017358914892122, + "rescued": 7, + "harmed": 698, + "missed_rescue": 1362, + "avoided_harm": 2223, + "selected_both_correct": 5930, + "selected_both_wrong": 52 + }, + { + "budget": 0.48, + "threshold": -0.7387455701828003, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.7059526319488368, + "rescued": 76, + "harmed": 707, + "missed_rescue": 1293, + "avoided_harm": 2214, + "selected_both_correct": 5945, + "selected_both_wrong": 101 + }, + { + "budget": 0.49, + "threshold": -0.7823225855827332, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.7123480216459344, + "rescued": 167, + "harmed": 707, + "missed_rescue": 1202, + "avoided_harm": 2214, + "selected_both_correct": 5945, + "selected_both_wrong": 153 + }, + { + "budget": 0.5, + "threshold": -0.8206579089164734, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.7179000632511069, + "rescued": 247, + "harmed": 708, + "missed_rescue": 1122, + "avoided_harm": 2213, + "selected_both_correct": 5945, + "selected_both_wrong": 214 + }, + { + "budget": 0.51, + "threshold": -0.8575630187988281, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.723381825848619, + "rescued": 326, + "harmed": 709, + "missed_rescue": 1043, + "avoided_harm": 2212, + "selected_both_correct": 5945, + "selected_both_wrong": 276 + }, + { + "budget": 0.52, + "threshold": -0.8823798894882202, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.729004146461452, + "rescued": 407, + "harmed": 710, + "missed_rescue": 962, + "avoided_harm": 2211, + "selected_both_correct": 5945, + "selected_both_wrong": 337 + }, + { + "budget": 0.53, + "threshold": -0.9040287733078003, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.7339939560053412, + "rescued": 478, + "harmed": 710, + "missed_rescue": 891, + "avoided_harm": 2211, + "selected_both_correct": 5946, + "selected_both_wrong": 407 + }, + { + "budget": 0.54, + "threshold": -0.9208687543869019, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.7390540445568908, + "rescued": 552, + "harmed": 712, + "missed_rescue": 817, + "avoided_harm": 2209, + "selected_both_correct": 5946, + "selected_both_wrong": 473 + }, + { + "budget": 0.55, + "threshold": -0.9344452619552612, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.743692459062478, + "rescued": 620, + "harmed": 714, + "missed_rescue": 749, + "avoided_harm": 2207, + "selected_both_correct": 5947, + "selected_both_wrong": 544 + }, + { + "budget": 0.56, + "threshold": -0.9448322653770447, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.7482605945604048, + "rescued": 690, + "harmed": 719, + "missed_rescue": 679, + "avoided_harm": 2202, + "selected_both_correct": 5948, + "selected_both_wrong": 611 + }, + { + "budget": 0.57, + "threshold": -0.9531905055046082, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.7525476140276899, + "rescued": 755, + "harmed": 723, + "missed_rescue": 614, + "avoided_harm": 2198, + "selected_both_correct": 5948, + "selected_both_wrong": 684 + }, + { + "budget": 0.58, + "threshold": -0.9603281617164612, + "operator": "gt", + "actual_fraction": 0.5798720922060581, + "metric": 0.7554993323494272, + "rescued": 802, + "harmed": 728, + "missed_rescue": 567, + "avoided_harm": 2193, + "selected_both_correct": 5949, + "selected_both_wrong": 772 + }, + { + "budget": 0.59, + "threshold": -0.9670801758766174, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.7591538407477686, + "rescued": 859, + "harmed": 733, + "missed_rescue": 510, + "avoided_harm": 2188, + "selected_both_correct": 5953, + "selected_both_wrong": 850 + }, + { + "budget": 0.6, + "threshold": -0.9724565148353577, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.7622461170848268, + "rescued": 911, + "harmed": 741, + "missed_rescue": 458, + "avoided_harm": 2180, + "selected_both_correct": 5956, + "selected_both_wrong": 929 + }, + { + "budget": 0.61, + "threshold": -0.9766654968261719, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.7658303464755077, + "rescued": 970, + "harmed": 749, + "missed_rescue": 399, + "avoided_harm": 2172, + "selected_both_correct": 5959, + "selected_both_wrong": 1001 + }, + { + "budget": 0.62, + "threshold": -0.9801702499389648, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.7682198327359617, + "rescued": 1014, + "harmed": 759, + "missed_rescue": 355, + "avoided_harm": 2162, + "selected_both_correct": 5961, + "selected_both_wrong": 1087 + }, + { + "budget": 0.63, + "threshold": -0.9831460118293762, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.769976807927472, + "rescued": 1049, + "harmed": 769, + "missed_rescue": 320, + "avoided_harm": 2152, + "selected_both_correct": 5971, + "selected_both_wrong": 1175 + }, + { + "budget": 0.64, + "threshold": -0.985751748085022, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.7713121090730198, + "rescued": 1089, + "harmed": 790, + "missed_rescue": 280, + "avoided_harm": 2131, + "selected_both_correct": 5977, + "selected_both_wrong": 1250 + }, + { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.7716635041113219, + "rescued": 1117, + "harmed": 813, + "missed_rescue": 252, + "avoided_harm": 2108, + "selected_both_correct": 5984, + "selected_both_wrong": 1334 + }, + { + "budget": 0.66, + "threshold": -0.9895955324172974, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.7709607140347178, + "rescued": 1141, + "harmed": 847, + "missed_rescue": 228, + "avoided_harm": 2074, + "selected_both_correct": 5993, + "selected_both_wrong": 1410 + }, + { + "budget": 0.67, + "threshold": -0.9913432002067566, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.7709607140347178, + "rescued": 1164, + "harmed": 870, + "missed_rescue": 205, + "avoided_harm": 2051, + "selected_both_correct": 6013, + "selected_both_wrong": 1486 + }, + { + "budget": 0.68, + "threshold": -0.9925730228424072, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.7703282029657741, + "rescued": 1191, + "harmed": 906, + "missed_rescue": 178, + "avoided_harm": 2015, + "selected_both_correct": 6033, + "selected_both_wrong": 1545 + }, + { + "budget": 0.69, + "threshold": -0.9935283660888672, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.7687117857895847, + "rescued": 1208, + "harmed": 946, + "missed_rescue": 161, + "avoided_harm": 1975, + "selected_both_correct": 6063, + "selected_both_wrong": 1601 + }, + { + "budget": 0.7, + "threshold": -0.99436354637146, + "operator": "gt", + "actual_fraction": 0.6999086372900415, + "metric": 0.7656195094525265, + "rescued": 1224, + "harmed": 1006, + "missed_rescue": 145, + "avoided_harm": 1915, + "selected_both_correct": 6082, + "selected_both_wrong": 1647 + }, + { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.7630191861690913, + "rescued": 1234, + "harmed": 1053, + "missed_rescue": 135, + "avoided_harm": 1868, + "selected_both_correct": 6118, + "selected_both_wrong": 1697 + }, + { + "budget": 0.72, + "threshold": -0.9955723881721497, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.7605594209009768, + "rescued": 1247, + "harmed": 1101, + "missed_rescue": 122, + "avoided_harm": 1820, + "selected_both_correct": 6158, + "selected_both_wrong": 1738 + }, + { + "budget": 0.73, + "threshold": -0.9960037469863892, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.7559210063953897, + "rescued": 1253, + "harmed": 1173, + "missed_rescue": 116, + "avoided_harm": 1748, + "selected_both_correct": 6185, + "selected_both_wrong": 1776 + }, + { + "budget": 0.74, + "threshold": -0.9964693188667297, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.7533206831119544, + "rescued": 1265, + "harmed": 1222, + "missed_rescue": 104, + "avoided_harm": 1699, + "selected_both_correct": 6233, + "selected_both_wrong": 1809 + }, + { + "budget": 0.75, + "threshold": -0.9968041777610779, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.7491742216599901, + "rescued": 1272, + "harmed": 1288, + "missed_rescue": 97, + "avoided_harm": 1633, + "selected_both_correct": 6272, + "selected_both_wrong": 1839 + }, + { + "budget": 0.76, + "threshold": -0.9971042275428772, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.7455899922693091, + "rescued": 1280, + "harmed": 1347, + "missed_rescue": 89, + "avoided_harm": 1574, + "selected_both_correct": 6315, + "selected_both_wrong": 1872 + }, + { + "budget": 0.77, + "threshold": -0.9974007606506348, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.7415840888326657, + "rescued": 1288, + "harmed": 1412, + "missed_rescue": 81, + "avoided_harm": 1509, + "selected_both_correct": 6353, + "selected_both_wrong": 1903 + }, + { + "budget": 0.78, + "threshold": -0.9976577162742615, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.7378593014266639, + "rescued": 1295, + "harmed": 1472, + "missed_rescue": 74, + "avoided_harm": 1449, + "selected_both_correct": 6399, + "selected_both_wrong": 1932 + }, + { + "budget": 0.79, + "threshold": -0.9978766441345215, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.7342750720359829, + "rescued": 1306, + "harmed": 1534, + "missed_rescue": 63, + "avoided_harm": 1387, + "selected_both_correct": 6446, + "selected_both_wrong": 1954 + }, + { + "budget": 0.8, + "threshold": -0.9980920553207397, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.7297069365380561, + "rescued": 1316, + "harmed": 1609, + "missed_rescue": 53, + "avoided_harm": 1312, + "selected_both_correct": 6491, + "selected_both_wrong": 1967 + }, + { + "budget": 0.81, + "threshold": -0.9982843399047852, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.7263335441703563, + "rescued": 1324, + "harmed": 1665, + "missed_rescue": 45, + "avoided_harm": 1256, + "selected_both_correct": 6553, + "selected_both_wrong": 1983 + }, + { + "budget": 0.82, + "threshold": -0.9984900951385498, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.7221168037107316, + "rescued": 1329, + "harmed": 1730, + "missed_rescue": 40, + "avoided_harm": 1191, + "selected_both_correct": 6599, + "selected_both_wrong": 2009 + }, + { + "budget": 0.83, + "threshold": -0.9986395835876465, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.718251458289409, + "rescued": 1333, + "harmed": 1789, + "missed_rescue": 36, + "avoided_harm": 1132, + "selected_both_correct": 6654, + "selected_both_wrong": 2034 + }, + { + "budget": 0.84, + "threshold": -0.9987816214561462, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.7131913697378593, + "rescued": 1337, + "harmed": 1865, + "missed_rescue": 32, + "avoided_harm": 1056, + "selected_both_correct": 6699, + "selected_both_wrong": 2051 + }, + { + "budget": 0.85, + "threshold": -0.9989162087440491, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.7093260243165367, + "rescued": 1340, + "harmed": 1923, + "missed_rescue": 29, + "avoided_harm": 998, + "selected_both_correct": 6763, + "selected_both_wrong": 2068 + }, + { + "budget": 0.86, + "threshold": -0.9990254044532776, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.7037037037037037, + "rescued": 1343, + "harmed": 2006, + "missed_rescue": 26, + "avoided_harm": 915, + "selected_both_correct": 6810, + "selected_both_wrong": 2077 + }, + { + "budget": 0.87, + "threshold": -0.9991487264633179, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.6985030571368332, + "rescued": 1347, + "harmed": 2084, + "missed_rescue": 22, + "avoided_harm": 837, + "selected_both_correct": 6856, + "selected_both_wrong": 2092 + }, + { + "budget": 0.88, + "threshold": -0.9992602467536926, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.6937943636235856, + "rescued": 1353, + "harmed": 2157, + "missed_rescue": 16, + "avoided_harm": 764, + "selected_both_correct": 6910, + "selected_both_wrong": 2101 + }, + { + "budget": 0.89, + "threshold": -0.999350905418396, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.6895776231639609, + "rescued": 1356, + "harmed": 2220, + "missed_rescue": 13, + "avoided_harm": 701, + "selected_both_correct": 6973, + "selected_both_wrong": 2114 + }, + { + "budget": 0.9, + "threshold": -0.999435544013977, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.6846580926277321, + "rescued": 1359, + "harmed": 2293, + "missed_rescue": 10, + "avoided_harm": 628, + "selected_both_correct": 7029, + "selected_both_wrong": 2125 + }, + { + "budget": 0.91, + "threshold": -0.9995256066322327, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.6794574460608617, + "rescued": 1359, + "harmed": 2367, + "missed_rescue": 10, + "avoided_harm": 554, + "selected_both_correct": 7084, + "selected_both_wrong": 2138 + }, + { + "budget": 0.92, + "threshold": -0.9995898604393005, + "operator": "ge", + "actual_fraction": 0.9199522102747909, + "metric": 0.6744676365169724, + "rescued": 1361, + "harmed": 2440, + "missed_rescue": 8, + "avoided_harm": 481, + "selected_both_correct": 7144, + "selected_both_wrong": 2145 + }, + { + "budget": 0.93, + "threshold": -0.9996523857116699, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.6701806170496872, + "rescued": 1364, + "harmed": 2504, + "missed_rescue": 5, + "avoided_harm": 417, + "selected_both_correct": 7216, + "selected_both_wrong": 2148 + }, + { + "budget": 0.94, + "threshold": -0.9997095465660095, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.6651908075057981, + "rescued": 1367, + "harmed": 2578, + "missed_rescue": 2, + "avoided_harm": 343, + "selected_both_correct": 7278, + "selected_both_wrong": 2152 + }, + { + "budget": 0.95, + "threshold": -0.9997634887695312, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.6606226720078713, + "rescued": 1367, + "harmed": 2643, + "missed_rescue": 2, + "avoided_harm": 278, + "selected_both_correct": 7349, + "selected_both_wrong": 2158 + }, + { + "budget": 0.96, + "threshold": -0.9998065829277039, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.6557031414716424, + "rescued": 1368, + "harmed": 2714, + "missed_rescue": 1, + "avoided_harm": 207, + "selected_both_correct": 7414, + "selected_both_wrong": 2163 + }, + { + "budget": 0.97, + "threshold": -0.9998431205749512, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.6521891910886218, + "rescued": 1368, + "harmed": 2764, + "missed_rescue": 1, + "avoided_harm": 157, + "selected_both_correct": 7498, + "selected_both_wrong": 2172 + }, + { + "budget": 0.98, + "threshold": -0.9998764991760254, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.6481832876519783, + "rescued": 1369, + "harmed": 2822, + "missed_rescue": 0, + "avoided_harm": 99, + "selected_both_correct": 7574, + "selected_both_wrong": 2179 + }, + { + "budget": 0.99, + "threshold": -0.9999123811721802, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.6447396162766181, + "rescued": 1369, + "harmed": 2871, + "missed_rescue": 0, + "avoided_harm": 50, + "selected_both_correct": 7664, + "selected_both_wrong": 2182 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.6412256658935975, + "rescued": 1369, + "harmed": 2921, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 7755, + "selected_both_wrong": 2184 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.6695119530385957, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2627, + "avoided_harm": 8602, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -2.0005727208172175e-07, + "operator": "ge", + "actual_fraction": 0.002467175226157729, + "metric": 0.6686044862887446, + "rescued": 0, + "harmed": 32, + "missed_rescue": 2627, + "avoided_harm": 8570, + "selected_both_correct": 55, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -2.4156264544217265e-07, + "operator": "ge", + "actual_fraction": 0.004849275444516916, + "metric": 0.6678388112185577, + "rescued": 0, + "harmed": 59, + "missed_rescue": 2627, + "avoided_harm": 8543, + "selected_both_correct": 112, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -2.7729177531909954e-07, + "operator": "ge", + "actual_fraction": 0.007600034030003119, + "metric": 0.667214927828035, + "rescued": 0, + "harmed": 81, + "missed_rescue": 2627, + "avoided_harm": 8521, + "selected_both_correct": 186, + "selected_both_wrong": 1 + }, + { + "budget": 0.04, + "threshold": -3.064858447032748e-07, + "operator": "ge", + "actual_fraction": 0.009811984232765221, + "metric": 0.6666761194453109, + "rescued": 0, + "harmed": 100, + "missed_rescue": 2627, + "avoided_harm": 8502, + "selected_both_correct": 245, + "selected_both_wrong": 1 + }, + { + "budget": 0.05, + "threshold": -3.3848826319626824e-07, + "operator": "ge", + "actual_fraction": 0.012534384482318578, + "metric": 0.6662791027422511, + "rescued": 0, + "harmed": 114, + "missed_rescue": 2627, + "avoided_harm": 8488, + "selected_both_correct": 326, + "selected_both_wrong": 2 + }, + { + "budget": 0.06, + "threshold": -3.668622525765386e-07, + "operator": "ge", + "actual_fraction": 0.01607917647392451, + "metric": 0.6658537277032583, + "rescued": 0, + "harmed": 129, + "missed_rescue": 2627, + "avoided_harm": 8473, + "selected_both_correct": 436, + "selected_both_wrong": 2 + }, + { + "budget": 0.07, + "threshold": -3.98026287484754e-07, + "operator": "ge", + "actual_fraction": 0.019709043473328986, + "metric": 0.6654850693361314, + "rescued": 0, + "harmed": 142, + "missed_rescue": 2627, + "avoided_harm": 8460, + "selected_both_correct": 550, + "selected_both_wrong": 3 + }, + { + "budget": 0.08, + "threshold": -4.302527258914779e-07, + "operator": "ge", + "actual_fraction": 0.02353741882426339, + "metric": 0.6652865609846014, + "rescued": 0, + "harmed": 149, + "missed_rescue": 2627, + "avoided_harm": 8453, + "selected_both_correct": 678, + "selected_both_wrong": 3 + }, + { + "budget": 0.09, + "threshold": -4.7191912244670675e-07, + "operator": "ge", + "actual_fraction": 0.028868785979638715, + "metric": 0.6648611859456087, + "rescued": 0, + "harmed": 164, + "missed_rescue": 2627, + "avoided_harm": 8438, + "selected_both_correct": 850, + "selected_both_wrong": 4 + }, + { + "budget": 0.1, + "threshold": -5.115426802149159e-07, + "operator": "ge", + "actual_fraction": 0.0345971698380739, + "metric": 0.6645208859144145, + "rescued": 0, + "harmed": 176, + "missed_rescue": 2627, + "avoided_harm": 8426, + "selected_both_correct": 1036, + "selected_both_wrong": 8 + }, + { + "budget": 0.11, + "threshold": -5.534793672268279e-07, + "operator": "ge", + "actual_fraction": 0.03981510364971783, + "metric": 0.6642089442191532, + "rescued": 0, + "harmed": 187, + "missed_rescue": 2627, + "avoided_harm": 8415, + "selected_both_correct": 1207, + "selected_both_wrong": 10 + }, + { + "budget": 0.12, + "threshold": -5.941800509390305e-07, + "operator": "ge", + "actual_fraction": 0.045855429203414344, + "metric": 0.6640387942035562, + "rescued": 0, + "harmed": 193, + "missed_rescue": 2627, + "avoided_harm": 8409, + "selected_both_correct": 1413, + "selected_both_wrong": 11 + }, + { + "budget": 0.13, + "threshold": -6.464504735959054e-07, + "operator": "ge", + "actual_fraction": 0.05317187987408899, + "metric": 0.6637268525082948, + "rescued": 1, + "harmed": 205, + "missed_rescue": 2626, + "avoided_harm": 8397, + "selected_both_correct": 1657, + "selected_both_wrong": 12 + }, + { + "budget": 0.14, + "threshold": -6.999837864896108e-07, + "operator": "ge", + "actual_fraction": 0.05960922213084536, + "metric": 0.6633581941411678, + "rescued": 1, + "harmed": 218, + "missed_rescue": 2626, + "avoided_harm": 8384, + "selected_both_correct": 1869, + "selected_both_wrong": 14 + }, + { + "budget": 0.15, + "threshold": -7.663418841730163e-07, + "operator": "ge", + "actual_fraction": 0.0675211978561098, + "metric": 0.6630178941099736, + "rescued": 1, + "harmed": 230, + "missed_rescue": 2626, + "avoided_harm": 8372, + "selected_both_correct": 2122, + "selected_both_wrong": 28 + }, + { + "budget": 0.16, + "threshold": -8.471382670904859e-07, + "operator": "ge", + "actual_fraction": 0.07551824858917279, + "metric": 0.6626492357428466, + "rescued": 1, + "harmed": 243, + "missed_rescue": 2626, + "avoided_harm": 8359, + "selected_both_correct": 2388, + "selected_both_wrong": 31 + }, + { + "budget": 0.17, + "threshold": -9.292756431023008e-07, + "operator": "ge", + "actual_fraction": 0.08212574086152624, + "metric": 0.6621671440319882, + "rescued": 1, + "harmed": 260, + "missed_rescue": 2626, + "avoided_harm": 8342, + "selected_both_correct": 2598, + "selected_both_wrong": 37 + }, + { + "budget": 0.18, + "threshold": -1.0245233852401725e-06, + "operator": "ge", + "actual_fraction": 0.08856308311828262, + "metric": 0.6619969940163911, + "rescued": 1, + "harmed": 266, + "missed_rescue": 2626, + "avoided_harm": 8336, + "selected_both_correct": 2814, + "selected_both_wrong": 42 + }, + { + "budget": 0.19, + "threshold": -1.1468601996966754e-06, + "operator": "ge", + "actual_fraction": 0.09480191702350906, + "metric": 0.6615716189773984, + "rescued": 1, + "harmed": 281, + "missed_rescue": 2626, + "avoided_harm": 8321, + "selected_both_correct": 3009, + "selected_both_wrong": 52 + }, + { + "budget": 0.2, + "threshold": -1.3018121762797819e-06, + "operator": "ge", + "actual_fraction": 0.10007656750701868, + "metric": 0.6612029606102714, + "rescued": 1, + "harmed": 294, + "missed_rescue": 2626, + "avoided_harm": 8308, + "selected_both_correct": 3174, + "selected_both_wrong": 60 + }, + { + "budget": 0.21, + "threshold": -1.4898060953782988e-06, + "operator": "ge", + "actual_fraction": 0.10554972634205825, + "metric": 0.6608910189150101, + "rescued": 1, + "harmed": 305, + "missed_rescue": 2626, + "avoided_harm": 8297, + "selected_both_correct": 3344, + "selected_both_wrong": 72 + }, + { + "budget": 0.22, + "threshold": -1.735574414851726e-06, + "operator": "ge", + "actual_fraction": 0.11005870175538099, + "metric": 0.6605790772197487, + "rescued": 1, + "harmed": 316, + "missed_rescue": 2626, + "avoided_harm": 8286, + "selected_both_correct": 3488, + "selected_both_wrong": 76 + }, + { + "budget": 0.23, + "threshold": -2.0881013824691763e-06, + "operator": "ge", + "actual_fraction": 0.11527663556702493, + "metric": 0.6602671355244875, + "rescued": 3, + "harmed": 329, + "missed_rescue": 2624, + "avoided_harm": 8273, + "selected_both_correct": 3638, + "selected_both_wrong": 95 + }, + { + "budget": 0.24, + "threshold": -2.5644872039265465e-06, + "operator": "ge", + "actual_fraction": 0.12029606102713893, + "metric": 0.6599835521651589, + "rescued": 4, + "harmed": 340, + "missed_rescue": 2623, + "avoided_harm": 8262, + "selected_both_correct": 3778, + "selected_both_wrong": 120 + }, + { + "budget": 0.25, + "threshold": -3.2399550491390983e-06, + "operator": "ge", + "actual_fraction": 0.12477667810452883, + "metric": 0.6596999688058305, + "rescued": 4, + "harmed": 350, + "missed_rescue": 2623, + "avoided_harm": 8252, + "selected_both_correct": 3906, + "selected_both_wrong": 140 + }, + { + "budget": 0.26, + "threshold": -4.2517635847616475e-06, + "operator": "ge", + "actual_fraction": 0.12914386183818732, + "metric": 0.659217877094972, + "rescued": 4, + "harmed": 367, + "missed_rescue": 2623, + "avoided_harm": 8235, + "selected_both_correct": 4023, + "selected_both_wrong": 160 + }, + { + "budget": 0.27, + "threshold": -5.607922958006384e-06, + "operator": "ge", + "actual_fraction": 0.1322916371267334, + "metric": 0.6587925020559794, + "rescued": 4, + "harmed": 382, + "missed_rescue": 2623, + "avoided_harm": 8220, + "selected_both_correct": 4102, + "selected_both_wrong": 177 + }, + { + "budget": 0.28, + "threshold": -8.438555596512742e-06, + "operator": "ge", + "actual_fraction": 0.13626180415733205, + "metric": 0.6582253353373224, + "rescued": 5, + "harmed": 403, + "missed_rescue": 2622, + "avoided_harm": 8199, + "selected_both_correct": 4197, + "selected_both_wrong": 200 + }, + { + "budget": 0.29, + "threshold": -1.3145449884177651e-05, + "operator": "ge", + "actual_fraction": 0.14023197118793068, + "metric": 0.6578283186342625, + "rescued": 6, + "harmed": 418, + "missed_rescue": 2621, + "avoided_harm": 8184, + "selected_both_correct": 4295, + "selected_both_wrong": 226 + }, + { + "budget": 0.3, + "threshold": -2.076632881653495e-05, + "operator": "ge", + "actual_fraction": 0.14326631313274538, + "metric": 0.6573178685874713, + "rescued": 7, + "harmed": 437, + "missed_rescue": 2620, + "avoided_harm": 8165, + "selected_both_correct": 4366, + "selected_both_wrong": 242 + }, + { + "budget": 0.31, + "threshold": -4.0114435250870883e-05, + "operator": "ge", + "actual_fraction": 0.14766185520233674, + "metric": 0.6569208518844114, + "rescued": 8, + "harmed": 452, + "missed_rescue": 2619, + "avoided_harm": 8150, + "selected_both_correct": 4472, + "selected_both_wrong": 275 + }, + { + "budget": 0.32, + "threshold": -7.415730942739174e-05, + "operator": "ge", + "actual_fraction": 0.15066783881121856, + "metric": 0.6565805518532173, + "rescued": 9, + "harmed": 465, + "missed_rescue": 2618, + "avoided_harm": 8137, + "selected_both_correct": 4550, + "selected_both_wrong": 289 + }, + { + "budget": 0.33, + "threshold": -0.00015223574882838875, + "operator": "ge", + "actual_fraction": 0.15452457249808582, + "metric": 0.6561268184782917, + "rescued": 12, + "harmed": 484, + "missed_rescue": 2615, + "avoided_harm": 8118, + "selected_both_correct": 4633, + "selected_both_wrong": 320 + }, + { + "budget": 0.34, + "threshold": -0.00032258269493468106, + "operator": "ge", + "actual_fraction": 0.15806936448969175, + "metric": 0.6558715934548961, + "rescued": 16, + "harmed": 497, + "missed_rescue": 2611, + "avoided_harm": 8105, + "selected_both_correct": 4709, + "selected_both_wrong": 352 + }, + { + "budget": 0.35, + "threshold": -0.0006007538759149611, + "operator": "ge", + "actual_fraction": 0.16201117318435754, + "metric": 0.6555596517596347, + "rescued": 17, + "harmed": 509, + "missed_rescue": 2610, + "avoided_harm": 8093, + "selected_both_correct": 4781, + "selected_both_wrong": 406 + }, + { + "budget": 0.36, + "threshold": -0.0011453087208792567, + "operator": "ge", + "actual_fraction": 0.1657544735274934, + "metric": 0.6551626350565749, + "rescued": 19, + "harmed": 525, + "missed_rescue": 2608, + "avoided_harm": 8077, + "selected_both_correct": 4844, + "selected_both_wrong": 457 + }, + { + "budget": 0.37, + "threshold": -0.002608026610687375, + "operator": "ge", + "actual_fraction": 0.17012165726115192, + "metric": 0.6545954683379179, + "rescued": 23, + "harmed": 549, + "missed_rescue": 2604, + "avoided_harm": 8053, + "selected_both_correct": 4930, + "selected_both_wrong": 497 + }, + { + "budget": 0.38, + "threshold": -0.006219128146767616, + "operator": "ge", + "actual_fraction": 0.17536794940872868, + "metric": 0.6540850182911266, + "rescued": 33, + "harmed": 577, + "missed_rescue": 2594, + "avoided_harm": 8025, + "selected_both_correct": 5012, + "selected_both_wrong": 562 + }, + { + "budget": 0.39, + "threshold": -0.01280172262340784, + "operator": "ge", + "actual_fraction": 0.17965005813458867, + "metric": 0.6534044182287383, + "rescued": 39, + "harmed": 607, + "missed_rescue": 2588, + "avoided_harm": 7995, + "selected_both_correct": 5077, + "selected_both_wrong": 612 + }, + { + "budget": 0.4, + "threshold": -0.03794781118631363, + "operator": "ge", + "actual_fraction": 0.18401724186824717, + "metric": 0.653092476533477, + "rescued": 44, + "harmed": 623, + "missed_rescue": 2583, + "avoided_harm": 7979, + "selected_both_correct": 5133, + "selected_both_wrong": 689 + }, + { + "budget": 0.41, + "threshold": -0.08479184657335281, + "operator": "ge", + "actual_fraction": 0.18858293395343562, + "metric": 0.6527521765022829, + "rescued": 46, + "harmed": 637, + "missed_rescue": 2581, + "avoided_harm": 7965, + "selected_both_correct": 5199, + "selected_both_wrong": 768 + }, + { + "budget": 0.42, + "threshold": -0.14941807091236115, + "operator": "ge", + "actual_fraction": 0.19238295096843716, + "metric": 0.6523835181351558, + "rescued": 52, + "harmed": 656, + "missed_rescue": 2575, + "avoided_harm": 7946, + "selected_both_correct": 5260, + "selected_both_wrong": 816 + }, + { + "budget": 0.43, + "threshold": -0.2939925789833069, + "operator": "ge", + "actual_fraction": 0.19689192638175992, + "metric": 0.6517596347446332, + "rescued": 63, + "harmed": 689, + "missed_rescue": 2564, + "avoided_harm": 7913, + "selected_both_correct": 5317, + "selected_both_wrong": 874 + }, + { + "budget": 0.44, + "threshold": -0.4386065602302551, + "operator": "ge", + "actual_fraction": 0.20182627683407536, + "metric": 0.6511641096900433, + "rescued": 77, + "harmed": 724, + "missed_rescue": 2550, + "avoided_harm": 7878, + "selected_both_correct": 5395, + "selected_both_wrong": 921 + }, + { + "budget": 0.45, + "threshold": -0.5420411825180054, + "operator": "ge", + "actual_fraction": 0.20718600232538353, + "metric": 0.6505402262995208, + "rescued": 97, + "harmed": 766, + "missed_rescue": 2530, + "avoided_harm": 7836, + "selected_both_correct": 5459, + "selected_both_wrong": 984 + }, + { + "budget": 0.46, + "threshold": -0.6230618953704834, + "operator": "ge", + "actual_fraction": 0.21189348609023623, + "metric": 0.6500864929245952, + "rescued": 122, + "harmed": 807, + "missed_rescue": 2505, + "avoided_harm": 7795, + "selected_both_correct": 5501, + "selected_both_wrong": 1042 + }, + { + "budget": 0.47, + "threshold": -0.6914289593696594, + "operator": "ge", + "actual_fraction": 0.217139778237813, + "metric": 0.6500581345886624, + "rescued": 151, + "harmed": 837, + "missed_rescue": 2476, + "avoided_harm": 7765, + "selected_both_correct": 5555, + "selected_both_wrong": 1114 + }, + { + "budget": 0.48, + "threshold": -0.7387455701828003, + "operator": "ge", + "actual_fraction": 0.22198905368232993, + "metric": 0.6507670929869835, + "rescued": 203, + "harmed": 864, + "missed_rescue": 2424, + "avoided_harm": 7738, + "selected_both_correct": 5574, + "selected_both_wrong": 1187 + }, + { + "budget": 0.49, + "threshold": -0.7823225855827332, + "operator": "ge", + "actual_fraction": 0.22763236253296656, + "metric": 0.6515611263931033, + "rescued": 267, + "harmed": 900, + "missed_rescue": 2360, + "avoided_harm": 7702, + "selected_both_correct": 5592, + "selected_both_wrong": 1268 + }, + { + "budget": 0.5, + "threshold": -0.8206579089164734, + "operator": "ge", + "actual_fraction": 0.2345234381646485, + "metric": 0.6527521765022829, + "rescued": 344, + "harmed": 935, + "missed_rescue": 2283, + "avoided_harm": 7667, + "selected_both_correct": 5617, + "selected_both_wrong": 1374 + }, + { + "budget": 0.51, + "threshold": -0.8575630187988281, + "operator": "ge", + "actual_fraction": 0.24382497235062248, + "metric": 0.6537447182599325, + "rescued": 430, + "harmed": 986, + "missed_rescue": 2197, + "avoided_harm": 7616, + "selected_both_correct": 5653, + "selected_both_wrong": 1529 + }, + { + "budget": 0.52, + "threshold": -0.8823798894882202, + "operator": "ge", + "actual_fraction": 0.252644414825738, + "metric": 0.6545387516660522, + "rescued": 501, + "harmed": 1029, + "missed_rescue": 2126, + "avoided_harm": 7573, + "selected_both_correct": 5683, + "selected_both_wrong": 1696 + }, + { + "budget": 0.53, + "threshold": -0.9040287733078003, + "operator": "ge", + "actual_fraction": 0.26401610753480986, + "metric": 0.6538581516036639, + "rescued": 571, + "harmed": 1123, + "missed_rescue": 2056, + "avoided_harm": 7479, + "selected_both_correct": 5730, + "selected_both_wrong": 1886 + }, + { + "budget": 0.54, + "threshold": -0.9208687543869019, + "operator": "ge", + "actual_fraction": 0.2760116836344043, + "metric": 0.6537163599239997, + "rescued": 647, + "harmed": 1204, + "missed_rescue": 1980, + "avoided_harm": 7398, + "selected_both_correct": 5804, + "selected_both_wrong": 2078 + }, + { + "budget": 0.55, + "threshold": -0.9344452619552612, + "operator": "ge", + "actual_fraction": 0.2893401015228426, + "metric": 0.6533477015568726, + "rescued": 728, + "harmed": 1298, + "missed_rescue": 1899, + "avoided_harm": 7304, + "selected_both_correct": 5894, + "selected_both_wrong": 2283 + }, + { + "budget": 0.56, + "threshold": -0.9448322653770447, + "operator": "ge", + "actual_fraction": 0.3036043444970649, + "metric": 0.6525536681507529, + "rescued": 811, + "harmed": 1409, + "missed_rescue": 1816, + "avoided_harm": 7193, + "selected_both_correct": 5983, + "selected_both_wrong": 2503 + }, + { + "budget": 0.57, + "threshold": -0.9531905055046082, + "operator": "ge", + "actual_fraction": 0.3175850041119587, + "metric": 0.6509088846666478, + "rescued": 876, + "harmed": 1532, + "missed_rescue": 1751, + "avoided_harm": 7070, + "selected_both_correct": 6081, + "selected_both_wrong": 2710 + }, + { + "budget": 0.58, + "threshold": -0.9603281617164612, + "operator": "gt", + "actual_fraction": 0.33462836400760004, + "metric": 0.6498312679011996, + "rescued": 971, + "harmed": 1665, + "missed_rescue": 1656, + "avoided_harm": 6937, + "selected_both_correct": 6236, + "selected_both_wrong": 2928 + }, + { + "budget": 0.59, + "threshold": -0.9670801758766174, + "operator": "ge", + "actual_fraction": 0.35637920766809406, + "metric": 0.6465984176048549, + "rescued": 1048, + "harmed": 1856, + "missed_rescue": 1579, + "avoided_harm": 6746, + "selected_both_correct": 6453, + "selected_both_wrong": 3210 + }, + { + "budget": 0.6, + "threshold": -0.9724565148353577, + "operator": "ge", + "actual_fraction": 0.37968975980489467, + "metric": 0.6442730340583614, + "rescued": 1153, + "harmed": 2043, + "missed_rescue": 1474, + "avoided_harm": 6559, + "selected_both_correct": 6702, + "selected_both_wrong": 3491 + }, + { + "budget": 0.61, + "threshold": -0.9766654968261719, + "operator": "ge", + "actual_fraction": 0.4046450954258004, + "metric": 0.6405580920511584, + "rescued": 1230, + "harmed": 2251, + "missed_rescue": 1397, + "avoided_harm": 6351, + "selected_both_correct": 6991, + "selected_both_wrong": 3797 + }, + { + "budget": 0.62, + "threshold": -0.9801702499389648, + "operator": "ge", + "actual_fraction": 0.43141536454640844, + "metric": 0.6357371749425744, + "rescued": 1303, + "harmed": 2494, + "missed_rescue": 1324, + "avoided_harm": 6108, + "selected_both_correct": 7308, + "selected_both_wrong": 4108 + }, + { + "budget": 0.63, + "threshold": -0.9831460118293762, + "operator": "ge", + "actual_fraction": 0.4610214672603012, + "metric": 0.630604316138729, + "rescued": 1384, + "harmed": 2756, + "missed_rescue": 1243, + "avoided_harm": 5846, + "selected_both_correct": 7664, + "selected_both_wrong": 4453 + }, + { + "budget": 0.64, + "threshold": -0.985751748085022, + "operator": "ge", + "actual_fraction": 0.4916484700677764, + "metric": 0.6244221989053682, + "rescued": 1476, + "harmed": 3066, + "missed_rescue": 1151, + "avoided_harm": 5536, + "selected_both_correct": 8035, + "selected_both_wrong": 4760 + }, + { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.5209426310864078, + "metric": 0.6187788900547316, + "rescued": 1563, + "harmed": 3352, + "missed_rescue": 1064, + "avoided_harm": 5250, + "selected_both_correct": 8439, + "selected_both_wrong": 5016 + }, + { + "budget": 0.66, + "threshold": -0.9895955324172974, + "operator": "ge", + "actual_fraction": 0.5529308340186598, + "metric": 0.6118310977511839, + "rescued": 1647, + "harmed": 3681, + "missed_rescue": 980, + "avoided_harm": 4921, + "selected_both_correct": 8872, + "selected_both_wrong": 5298 + }, + { + "budget": 0.67, + "threshold": -0.9913432002067566, + "operator": "ge", + "actual_fraction": 0.5888041289737118, + "metric": 0.6045146470805093, + "rescued": 1741, + "harmed": 4033, + "missed_rescue": 886, + "avoided_harm": 4569, + "selected_both_correct": 9334, + "selected_both_wrong": 5655 + }, + { + "budget": 0.68, + "threshold": -0.9925730228424072, + "operator": "ge", + "actual_fraction": 0.6175027649377535, + "metric": 0.5983608881830814, + "rescued": 1805, + "harmed": 4314, + "missed_rescue": 822, + "avoided_harm": 4288, + "selected_both_correct": 9696, + "selected_both_wrong": 5960 + }, + { + "budget": 0.69, + "threshold": -0.9935283660888672, + "operator": "ge", + "actual_fraction": 0.6421461588633979, + "metric": 0.5925474293168477, + "rescued": 1860, + "harmed": 4574, + "missed_rescue": 767, + "avoided_harm": 4028, + "selected_both_correct": 10008, + "selected_both_wrong": 6202 + }, + { + "budget": 0.7, + "threshold": -0.99436354637146, + "operator": "gt", + "actual_fraction": 0.665910444375124, + "metric": 0.5877548705441965, + "rescued": 1925, + "harmed": 4808, + "missed_rescue": 702, + "avoided_harm": 3794, + "selected_both_correct": 10303, + "selected_both_wrong": 6446 + }, + { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.6838612710206166, + "metric": 0.5829339534356124, + "rescued": 1961, + "harmed": 5014, + "missed_rescue": 666, + "avoided_harm": 3588, + "selected_both_correct": 10525, + "selected_both_wrong": 6615 + }, + { + "budget": 0.72, + "threshold": -0.9955723881721497, + "operator": "ge", + "actual_fraction": 0.7029464311034228, + "metric": 0.5779996029832969, + "rescued": 2000, + "harmed": 5227, + "missed_rescue": 627, + "avoided_harm": 3375, + "selected_both_correct": 10747, + "selected_both_wrong": 6814 + }, + { + "budget": 0.73, + "threshold": -0.9960037469863892, + "operator": "ge", + "actual_fraction": 0.7180614241556306, + "metric": 0.5745966026713553, + "rescued": 2042, + "harmed": 5389, + "missed_rescue": 585, + "avoided_harm": 3213, + "selected_both_correct": 10923, + "selected_both_wrong": 6967 + }, + { + "budget": 0.74, + "threshold": -0.9964693188667297, + "operator": "ge", + "actual_fraction": 0.7352465757309361, + "metric": 0.5710234523438165, + "rescued": 2087, + "harmed": 5560, + "missed_rescue": 540, + "avoided_harm": 3042, + "selected_both_correct": 11139, + "selected_both_wrong": 7141 + }, + { + "budget": 0.75, + "threshold": -0.9968041777610779, + "operator": "ge", + "actual_fraction": 0.7478093185491875, + "metric": 0.567762243711539, + "rescued": 2117, + "harmed": 5705, + "missed_rescue": 510, + "avoided_harm": 2897, + "selected_both_correct": 11290, + "selected_both_wrong": 7258 + }, + { + "budget": 0.76, + "threshold": -0.9971042275428772, + "operator": "ge", + "actual_fraction": 0.759804894648782, + "metric": 0.565153276805717, + "rescued": 2149, + "harmed": 5829, + "missed_rescue": 478, + "avoided_harm": 2773, + "selected_both_correct": 11421, + "selected_both_wrong": 7394 + }, + { + "budget": 0.77, + "threshold": -0.9974007606506348, + "operator": "ge", + "actual_fraction": 0.7729064458497575, + "metric": 0.5620622181890367, + "rescued": 2183, + "harmed": 5972, + "missed_rescue": 444, + "avoided_harm": 2630, + "selected_both_correct": 11577, + "selected_both_wrong": 7523 + }, + { + "budget": 0.78, + "threshold": -0.9976577162742615, + "operator": "ge", + "actual_fraction": 0.7855259053398747, + "metric": 0.5588860845645578, + "rescued": 2221, + "harmed": 6122, + "missed_rescue": 406, + "avoided_harm": 2480, + "selected_both_correct": 11724, + "selected_both_wrong": 7633 + }, + { + "budget": 0.79, + "threshold": -0.9978766441345215, + "operator": "ge", + "actual_fraction": 0.7964722230099538, + "metric": 0.5562771176587358, + "rescued": 2244, + "harmed": 6237, + "missed_rescue": 383, + "avoided_harm": 2365, + "selected_both_correct": 11855, + "selected_both_wrong": 7750 + }, + { + "budget": 0.8, + "threshold": -0.9980920553207397, + "operator": "ge", + "actual_fraction": 0.8082125740861527, + "metric": 0.5536114340810481, + "rescued": 2274, + "harmed": 6361, + "missed_rescue": 353, + "avoided_harm": 2241, + "selected_both_correct": 12000, + "selected_both_wrong": 7865 + }, + { + "budget": 0.81, + "threshold": -0.9982843399047852, + "operator": "ge", + "actual_fraction": 0.8197544168108215, + "metric": 0.5504920171284349, + "rescued": 2300, + "harmed": 6497, + "missed_rescue": 327, + "avoided_harm": 2105, + "selected_both_correct": 12140, + "selected_both_wrong": 7970 + }, + { + "budget": 0.82, + "threshold": -0.9984900951385498, + "operator": "ge", + "actual_fraction": 0.8333380597226555, + "metric": 0.5469188668008961, + "rescued": 2329, + "harmed": 6652, + "missed_rescue": 298, + "avoided_harm": 1950, + "selected_both_correct": 12321, + "selected_both_wrong": 8084 + }, + { + "budget": 0.83, + "threshold": -0.9986395835876465, + "operator": "ge", + "actual_fraction": 0.8426112355726966, + "metric": 0.5450188582933954, + "rescued": 2356, + "harmed": 6746, + "missed_rescue": 271, + "avoided_harm": 1856, + "selected_both_correct": 12437, + "selected_both_wrong": 8174 + }, + { + "budget": 0.84, + "threshold": -0.9987816214561462, + "operator": "ge", + "actual_fraction": 0.8529903865241187, + "metric": 0.5423531747157077, + "rescued": 2381, + "harmed": 6865, + "missed_rescue": 246, + "avoided_harm": 1737, + "selected_both_correct": 12586, + "selected_both_wrong": 8247 + }, + { + "budget": 0.85, + "threshold": -0.9989162087440491, + "operator": "ge", + "actual_fraction": 0.8635396874911381, + "metric": 0.5395456994583557, + "rescued": 2414, + "harmed": 6997, + "missed_rescue": 213, + "avoided_harm": 1605, + "selected_both_correct": 12707, + "selected_both_wrong": 8333 + }, + { + "budget": 0.86, + "threshold": -0.9990254044532776, + "operator": "ge", + "actual_fraction": 0.8710262881774098, + "metric": 0.5378725576383178, + "rescued": 2435, + "harmed": 7077, + "missed_rescue": 192, + "avoided_harm": 1525, + "selected_both_correct": 12808, + "selected_both_wrong": 8395 + }, + { + "budget": 0.87, + "threshold": -0.9991487264633179, + "operator": "ge", + "actual_fraction": 0.8809517057539064, + "metric": 0.535774040779287, + "rescued": 2462, + "harmed": 7178, + "missed_rescue": 165, + "avoided_harm": 1424, + "selected_both_correct": 12942, + "selected_both_wrong": 8483 + }, + { + "budget": 0.88, + "threshold": -0.9992602467536926, + "operator": "ge", + "actual_fraction": 0.8913024983693957, + "metric": 0.5330516405297338, + "rescued": 2480, + "harmed": 7292, + "missed_rescue": 147, + "avoided_harm": 1310, + "selected_both_correct": 13082, + "selected_both_wrong": 8576 + }, + { + "budget": 0.89, + "threshold": -0.999350905418396, + "operator": "ge", + "actual_fraction": 0.900915974250631, + "metric": 0.5304710319598446, + "rescued": 2498, + "harmed": 7401, + "missed_rescue": 129, + "avoided_harm": 1201, + "selected_both_correct": 13224, + "selected_both_wrong": 8646 + }, + { + "budget": 0.9, + "threshold": -0.999435544013977, + "operator": "ge", + "actual_fraction": 0.910954825170859, + "metric": 0.5277486317102913, + "rescued": 2518, + "harmed": 7517, + "missed_rescue": 109, + "avoided_harm": 1085, + "selected_both_correct": 13379, + "selected_both_wrong": 8709 + }, + { + "budget": 0.91, + "threshold": -0.9995256066322327, + "operator": "ge", + "actual_fraction": 0.921901142840938, + "metric": 0.5245724980858123, + "rescued": 2542, + "harmed": 7653, + "missed_rescue": 85, + "avoided_harm": 949, + "selected_both_correct": 13541, + "selected_both_wrong": 8773 + }, + { + "budget": 0.92, + "threshold": -0.9995898604393005, + "operator": "ge", + "actual_fraction": 0.9303235686129938, + "metric": 0.5217366644925275, + "rescued": 2553, + "harmed": 7764, + "missed_rescue": 74, + "avoided_harm": 838, + "selected_both_correct": 13673, + "selected_both_wrong": 8816 + }, + { + "budget": 0.93, + "threshold": -0.9996523857116699, + "operator": "ge", + "actual_fraction": 0.9393698777755721, + "metric": 0.5187306808836457, + "rescued": 2567, + "harmed": 7884, + "missed_rescue": 60, + "avoided_harm": 718, + "selected_both_correct": 13819, + "selected_both_wrong": 8855 + }, + { + "budget": 0.94, + "threshold": -0.9997095465660095, + "operator": "ge", + "actual_fraction": 0.9490117119927403, + "metric": 0.515526188923234, + "rescued": 2578, + "harmed": 8008, + "missed_rescue": 49, + "avoided_harm": 594, + "selected_both_correct": 13977, + "selected_both_wrong": 8902 + }, + { + "budget": 0.95, + "threshold": -0.9997634887695312, + "operator": "ge", + "actual_fraction": 0.9587953378895726, + "metric": 0.5122366219550237, + "rescued": 2588, + "harmed": 8134, + "missed_rescue": 39, + "avoided_harm": 468, + "selected_both_correct": 14140, + "selected_both_wrong": 8948 + }, + { + "budget": 0.96, + "threshold": -0.9998065829277039, + "operator": "ge", + "actual_fraction": 0.9673595553412926, + "metric": 0.5097694467288659, + "rescued": 2604, + "harmed": 8237, + "missed_rescue": 23, + "avoided_harm": 365, + "selected_both_correct": 14295, + "selected_both_wrong": 8976 + }, + { + "budget": 0.97, + "threshold": -0.9998431205749512, + "operator": "ge", + "actual_fraction": 0.9746192893401016, + "metric": 0.5075574965261038, + "rescued": 2615, + "harmed": 8326, + "missed_rescue": 12, + "avoided_harm": 276, + "selected_both_correct": 14432, + "selected_both_wrong": 8995 + }, + { + "budget": 0.98, + "threshold": -0.9998764991760254, + "operator": "ge", + "actual_fraction": 0.9830984317840229, + "metric": 0.5046365879250205, + "rescued": 2623, + "harmed": 8437, + "missed_rescue": 4, + "avoided_harm": 165, + "selected_both_correct": 14600, + "selected_both_wrong": 9007 + }, + { + "budget": 0.99, + "threshold": -0.9999123811721802, + "operator": "ge", + "actual_fraction": 0.9908118991577575, + "metric": 0.5019425460114001, + "rescued": 2626, + "harmed": 8535, + "missed_rescue": 1, + "avoided_harm": 67, + "selected_both_correct": 14761, + "selected_both_wrong": 9017 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5000708958398321, + "rescued": 2627, + "harmed": 8602, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 15007, + "selected_both_wrong": 9027 + } + ] + }, + "learned": { + "selected_budget_index": 23, + "validation_area_0_25": 0.7607140347178298, + "validation_point": { + "budget": 0.23, + "threshold": -5.381783485412598, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.7729988052568698, + "rescued": 773, + "harmed": 450, + "missed_rescue": 596, + "avoided_harm": 2471, + "selected_both_correct": 843, + "selected_both_wrong": 1206 + }, + "test_point": { + "budget": 0.23, + "threshold": -5.381783485412598, + "operator": "ge", + "actual_fraction": 0.30904914499617164, + "metric": 0.6671865694921022, + "rescued": 1570, + "harmed": 1652, + "missed_rescue": 1057, + "avoided_harm": 6950, + "selected_both_correct": 1879, + "selected_both_wrong": 5797 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7502986857825568, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1369, + "avoided_harm": 2921, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.430821418762207, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.7526881720430108, + "rescued": 38, + "harmed": 4, + "missed_rescue": 1331, + "avoided_harm": 2917, + "selected_both_correct": 21, + "selected_both_wrong": 79 + }, + { + "budget": 0.02, + "threshold": 0.8396652936935425, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.7528287300583316, + "rescued": 59, + "harmed": 23, + "missed_rescue": 1310, + "avoided_harm": 2898, + "selected_both_correct": 71, + "selected_both_wrong": 131 + }, + { + "budget": 0.03, + "threshold": -0.16024979948997498, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.7518448239510858, + "rescued": 70, + "harmed": 48, + "missed_rescue": 1299, + "avoided_harm": 2873, + "selected_both_correct": 129, + "selected_both_wrong": 179 + }, + { + "budget": 0.04, + "threshold": -0.9098284244537354, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.7521962189893878, + "rescued": 92, + "harmed": 65, + "missed_rescue": 1277, + "avoided_harm": 2856, + "selected_both_correct": 177, + "selected_both_wrong": 235 + }, + { + "budget": 0.05, + "threshold": -1.4125885963439941, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.7518448239510858, + "rescued": 113, + "harmed": 91, + "missed_rescue": 1256, + "avoided_harm": 2830, + "selected_both_correct": 214, + "selected_both_wrong": 293 + }, + { + "budget": 0.06, + "threshold": -1.8043409585952759, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.7523367770047087, + "rescued": 137, + "harmed": 108, + "missed_rescue": 1232, + "avoided_harm": 2813, + "selected_both_correct": 265, + "selected_both_wrong": 343 + }, + { + "budget": 0.07, + "threshold": -2.2280092239379883, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.7530395670813128, + "rescued": 170, + "harmed": 131, + "missed_rescue": 1199, + "avoided_harm": 2790, + "selected_both_correct": 297, + "selected_both_wrong": 398 + }, + { + "budget": 0.08, + "threshold": -2.5369863510131836, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.754093752196219, + "rescued": 196, + "harmed": 142, + "missed_rescue": 1173, + "avoided_harm": 2779, + "selected_both_correct": 331, + "selected_both_wrong": 469 + }, + { + "budget": 0.09, + "threshold": -2.807616949081421, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.754937100288144, + "rescued": 233, + "harmed": 167, + "missed_rescue": 1136, + "avoided_harm": 2754, + "selected_both_correct": 359, + "selected_both_wrong": 521 + }, + { + "budget": 0.1, + "threshold": -3.0437393188476562, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.7566237964719938, + "rescued": 272, + "harmed": 182, + "missed_rescue": 1097, + "avoided_harm": 2739, + "selected_both_correct": 384, + "selected_both_wrong": 584 + }, + { + "budget": 0.11, + "threshold": -3.299111843109131, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.7583807716635041, + "rescued": 319, + "harmed": 204, + "missed_rescue": 1050, + "avoided_harm": 2717, + "selected_both_correct": 413, + "selected_both_wrong": 629 + }, + { + "budget": 0.12, + "threshold": -3.5267717838287354, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.7603485838779956, + "rescued": 359, + "harmed": 216, + "missed_rescue": 1010, + "avoided_harm": 2705, + "selected_both_correct": 441, + "selected_both_wrong": 691 + }, + { + "budget": 0.13, + "threshold": -3.745974540710449, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.761613606015883, + "rescued": 395, + "harmed": 234, + "missed_rescue": 974, + "avoided_harm": 2687, + "selected_both_correct": 463, + "selected_both_wrong": 757 + }, + { + "budget": 0.14, + "threshold": -3.9486825466156006, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.7631597441844121, + "rescued": 432, + "harmed": 249, + "missed_rescue": 937, + "avoided_harm": 2672, + "selected_both_correct": 494, + "selected_both_wrong": 817 + }, + { + "budget": 0.15, + "threshold": -4.13905668258667, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.7641436502916579, + "rescued": 468, + "harmed": 271, + "missed_rescue": 901, + "avoided_harm": 2650, + "selected_both_correct": 519, + "selected_both_wrong": 876 + }, + { + "budget": 0.16, + "threshold": -4.306515693664551, + "operator": "ge", + "actual_fraction": 0.15995502143509735, + "metric": 0.7654086724295452, + "rescued": 505, + "harmed": 290, + "missed_rescue": 864, + "avoided_harm": 2631, + "selected_both_correct": 558, + "selected_both_wrong": 923 + }, + { + "budget": 0.17, + "threshold": -4.4712700843811035, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.767235926628716, + "rescued": 551, + "harmed": 310, + "missed_rescue": 818, + "avoided_harm": 2611, + "selected_both_correct": 591, + "selected_both_wrong": 966 + }, + { + "budget": 0.18, + "threshold": -4.667325019836426, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.768079274720641, + "rescued": 591, + "harmed": 338, + "missed_rescue": 778, + "avoided_harm": 2583, + "selected_both_correct": 623, + "selected_both_wrong": 1009 + }, + { + "budget": 0.19, + "threshold": -4.808548450469971, + "operator": "gt", + "actual_fraction": 0.18989387869843277, + "metric": 0.7678684376976597, + "rescued": 618, + "harmed": 368, + "missed_rescue": 751, + "avoided_harm": 2553, + "selected_both_correct": 659, + "selected_both_wrong": 1057 + }, + { + "budget": 0.2, + "threshold": -4.945236682891846, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.7694848548738492, + "rescued": 658, + "harmed": 385, + "missed_rescue": 711, + "avoided_harm": 2536, + "selected_both_correct": 708, + "selected_both_wrong": 1094 + }, + { + "budget": 0.21, + "threshold": -5.0980048179626465, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.7709607140347178, + "rescued": 696, + "harmed": 402, + "missed_rescue": 673, + "avoided_harm": 2519, + "selected_both_correct": 755, + "selected_both_wrong": 1135 + }, + { + "budget": 0.22, + "threshold": -5.24181604385376, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.7718040621266428, + "rescued": 731, + "harmed": 425, + "missed_rescue": 638, + "avoided_harm": 2496, + "selected_both_correct": 800, + "selected_both_wrong": 1174 + }, + { + "budget": 0.23, + "threshold": -5.381783485412598, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.7729988052568698, + "rescued": 773, + "harmed": 450, + "missed_rescue": 596, + "avoided_harm": 2471, + "selected_both_correct": 843, + "selected_both_wrong": 1206 + }, + { + "budget": 0.24, + "threshold": -5.480440616607666, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.7727176892262281, + "rescued": 796, + "harmed": 477, + "missed_rescue": 573, + "avoided_harm": 2444, + "selected_both_correct": 898, + "selected_both_wrong": 1243 + }, + { + "budget": 0.25, + "threshold": -5.592668533325195, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.7726474102185678, + "rescued": 823, + "harmed": 505, + "missed_rescue": 546, + "avoided_harm": 2416, + "selected_both_correct": 951, + "selected_both_wrong": 1278 + }, + { + "budget": 0.26, + "threshold": -5.701513290405273, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.7725068522032469, + "rescued": 849, + "harmed": 533, + "missed_rescue": 520, + "avoided_harm": 2388, + "selected_both_correct": 1019, + "selected_both_wrong": 1298 + }, + { + "budget": 0.27, + "threshold": -5.817866802215576, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.7726474102185678, + "rescued": 879, + "harmed": 561, + "missed_rescue": 490, + "avoided_harm": 2360, + "selected_both_correct": 1077, + "selected_both_wrong": 1324 + }, + { + "budget": 0.28, + "threshold": -5.915761947631836, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.7726474102185678, + "rescued": 901, + "harmed": 583, + "missed_rescue": 468, + "avoided_harm": 2338, + "selected_both_correct": 1152, + "selected_both_wrong": 1348 + }, + { + "budget": 0.29, + "threshold": -6.032350063323975, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.7715932251036616, + "rescued": 915, + "harmed": 612, + "missed_rescue": 454, + "avoided_harm": 2309, + "selected_both_correct": 1218, + "selected_both_wrong": 1381 + }, + { + "budget": 0.3, + "threshold": -6.134230136871338, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.7706093189964157, + "rescued": 930, + "harmed": 641, + "missed_rescue": 439, + "avoided_harm": 2280, + "selected_both_correct": 1281, + "selected_both_wrong": 1416 + }, + { + "budget": 0.31, + "threshold": -6.218080520629883, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.7701173659427929, + "rescued": 954, + "harmed": 672, + "missed_rescue": 415, + "avoided_harm": 2249, + "selected_both_correct": 1336, + "selected_both_wrong": 1448 + }, + { + "budget": 0.32, + "threshold": -6.302128791809082, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.7697659709044908, + "rescued": 972, + "harmed": 695, + "missed_rescue": 397, + "avoided_harm": 2226, + "selected_both_correct": 1412, + "selected_both_wrong": 1474 + }, + { + "budget": 0.33, + "threshold": -6.377316474914551, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.7685712277742638, + "rescued": 986, + "harmed": 726, + "missed_rescue": 383, + "avoided_harm": 2195, + "selected_both_correct": 1485, + "selected_both_wrong": 1498 + }, + { + "budget": 0.34, + "threshold": -6.449829578399658, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.768430669758943, + "rescued": 1008, + "harmed": 750, + "missed_rescue": 361, + "avoided_harm": 2171, + "selected_both_correct": 1563, + "selected_both_wrong": 1516 + }, + { + "budget": 0.35, + "threshold": -6.539102077484131, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.7677981586899993, + "rescued": 1025, + "harmed": 776, + "missed_rescue": 344, + "avoided_harm": 2145, + "selected_both_correct": 1641, + "selected_both_wrong": 1538 + }, + { + "budget": 0.36, + "threshold": -6.614948272705078, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.7666034155597723, + "rescued": 1035, + "harmed": 803, + "missed_rescue": 334, + "avoided_harm": 2118, + "selected_both_correct": 1725, + "selected_both_wrong": 1559 + }, + { + "budget": 0.37, + "threshold": -6.686672210693359, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.7658303464755077, + "rescued": 1047, + "harmed": 826, + "missed_rescue": 322, + "avoided_harm": 2095, + "selected_both_correct": 1797, + "selected_both_wrong": 1594 + }, + { + "budget": 0.38, + "threshold": -6.78654146194458, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.7642139292993183, + "rescued": 1056, + "harmed": 858, + "missed_rescue": 313, + "avoided_harm": 2063, + "selected_both_correct": 1876, + "selected_both_wrong": 1617 + }, + { + "budget": 0.39, + "threshold": -6.846216201782227, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.7630894651767517, + "rescued": 1067, + "harmed": 885, + "missed_rescue": 302, + "avoided_harm": 2036, + "selected_both_correct": 1960, + "selected_both_wrong": 1637 + }, + { + "budget": 0.4, + "threshold": -6.915761470794678, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.7618947220465248, + "rescued": 1083, + "harmed": 918, + "missed_rescue": 286, + "avoided_harm": 2003, + "selected_both_correct": 2035, + "selected_both_wrong": 1655 + }, + { + "budget": 0.41, + "threshold": -6.9817047119140625, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.7602783048703352, + "rescued": 1096, + "harmed": 954, + "missed_rescue": 273, + "avoided_harm": 1967, + "selected_both_correct": 2111, + "selected_both_wrong": 1672 + }, + { + "budget": 0.42, + "threshold": -7.047516822814941, + "operator": "gt", + "actual_fraction": 0.4198467917633003, + "metric": 0.7592943987630895, + "rescued": 1106, + "harmed": 978, + "missed_rescue": 263, + "avoided_harm": 1943, + "selected_both_correct": 2195, + "selected_both_wrong": 1695 + }, + { + "budget": 0.43, + "threshold": -7.1213059425354, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.7578185396022208, + "rescued": 1114, + "harmed": 1007, + "missed_rescue": 255, + "avoided_harm": 1914, + "selected_both_correct": 2279, + "selected_both_wrong": 1718 + }, + { + "budget": 0.44, + "threshold": -7.181412696838379, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.7569049125026355, + "rescued": 1125, + "harmed": 1031, + "missed_rescue": 244, + "avoided_harm": 1890, + "selected_both_correct": 2363, + "selected_both_wrong": 1741 + }, + { + "budget": 0.45, + "threshold": -7.25251579284668, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.7559210063953897, + "rescued": 1135, + "harmed": 1055, + "missed_rescue": 234, + "avoided_harm": 1866, + "selected_both_correct": 2455, + "selected_both_wrong": 1758 + }, + { + "budget": 0.46, + "threshold": -7.320802211761475, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.7554290533417668, + "rescued": 1144, + "harmed": 1071, + "missed_rescue": 225, + "avoided_harm": 1850, + "selected_both_correct": 2558, + "selected_both_wrong": 1772 + }, + { + "budget": 0.47, + "threshold": -7.374159812927246, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.7533206831119544, + "rescued": 1148, + "harmed": 1105, + "missed_rescue": 221, + "avoided_harm": 1816, + "selected_both_correct": 2651, + "selected_both_wrong": 1783 + }, + { + "budget": 0.48, + "threshold": -7.4538164138793945, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.7522664979970483, + "rescued": 1156, + "harmed": 1128, + "missed_rescue": 213, + "avoided_harm": 1793, + "selected_both_correct": 2751, + "selected_both_wrong": 1794 + }, + { + "budget": 0.49, + "threshold": -7.532625675201416, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.7510014758591609, + "rescued": 1162, + "harmed": 1152, + "missed_rescue": 207, + "avoided_harm": 1769, + "selected_both_correct": 2852, + "selected_both_wrong": 1806 + }, + { + "budget": 0.5, + "threshold": -7.600386619567871, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.7501581277672359, + "rescued": 1172, + "harmed": 1174, + "missed_rescue": 197, + "avoided_harm": 1747, + "selected_both_correct": 2947, + "selected_both_wrong": 1821 + }, + { + "budget": 0.51, + "threshold": -7.662148952484131, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.7490336636446694, + "rescued": 1183, + "harmed": 1201, + "missed_rescue": 186, + "avoided_harm": 1720, + "selected_both_correct": 3036, + "selected_both_wrong": 1836 + }, + { + "budget": 0.52, + "threshold": -7.733202934265137, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.7479091995221028, + "rescued": 1192, + "harmed": 1226, + "missed_rescue": 177, + "avoided_harm": 1695, + "selected_both_correct": 3132, + "selected_both_wrong": 1849 + }, + { + "budget": 0.53, + "threshold": -7.79387092590332, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.7471361304378382, + "rescued": 1203, + "harmed": 1248, + "missed_rescue": 166, + "avoided_harm": 1673, + "selected_both_correct": 3232, + "selected_both_wrong": 1858 + }, + { + "budget": 0.54, + "threshold": -7.855954647064209, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.7453088762386675, + "rescued": 1211, + "harmed": 1282, + "missed_rescue": 158, + "avoided_harm": 1639, + "selected_both_correct": 3315, + "selected_both_wrong": 1875 + }, + { + "budget": 0.55, + "threshold": -7.925965785980225, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.7439032960854592, + "rescued": 1219, + "harmed": 1310, + "missed_rescue": 150, + "avoided_harm": 1611, + "selected_both_correct": 3410, + "selected_both_wrong": 1886 + }, + { + "budget": 0.56, + "threshold": -7.977593421936035, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.7429193899782135, + "rescued": 1229, + "harmed": 1334, + "missed_rescue": 140, + "avoided_harm": 1587, + "selected_both_correct": 3507, + "selected_both_wrong": 1898 + }, + { + "budget": 0.57, + "threshold": -8.031231880187988, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.7415138098250053, + "rescued": 1240, + "harmed": 1365, + "missed_rescue": 129, + "avoided_harm": 1556, + "selected_both_correct": 3597, + "selected_both_wrong": 1908 + }, + { + "budget": 0.58, + "threshold": -8.088851928710938, + "operator": "ge", + "actual_fraction": 0.5799423712137185, + "metric": 0.7399676716564763, + "rescued": 1245, + "harmed": 1392, + "missed_rescue": 124, + "avoided_harm": 1529, + "selected_both_correct": 3698, + "selected_both_wrong": 1917 + }, + { + "budget": 0.59, + "threshold": -8.145581245422363, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.7384215334879471, + "rescued": 1251, + "harmed": 1420, + "missed_rescue": 118, + "avoided_harm": 1501, + "selected_both_correct": 3793, + "selected_both_wrong": 1931 + }, + { + "budget": 0.6, + "threshold": -8.199271202087402, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.7371565113500598, + "rescued": 1254, + "harmed": 1441, + "missed_rescue": 115, + "avoided_harm": 1480, + "selected_both_correct": 3894, + "selected_both_wrong": 1948 + }, + { + "budget": 0.61, + "threshold": -8.256582260131836, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.7357509311968515, + "rescued": 1260, + "harmed": 1467, + "missed_rescue": 109, + "avoided_harm": 1454, + "selected_both_correct": 3992, + "selected_both_wrong": 1960 + }, + { + "budget": 0.62, + "threshold": -8.316590309143066, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.7338533979900204, + "rescued": 1265, + "harmed": 1499, + "missed_rescue": 104, + "avoided_harm": 1422, + "selected_both_correct": 4091, + "selected_both_wrong": 1966 + }, + { + "budget": 0.63, + "threshold": -8.380763053894043, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.731745027760208, + "rescued": 1271, + "harmed": 1535, + "missed_rescue": 98, + "avoided_harm": 1386, + "selected_both_correct": 4186, + "selected_both_wrong": 1972 + }, + { + "budget": 0.64, + "threshold": -8.436734199523926, + "operator": "gt", + "actual_fraction": 0.6398903647480497, + "metric": 0.7303394476069998, + "rescued": 1277, + "harmed": 1561, + "missed_rescue": 92, + "avoided_harm": 1360, + "selected_both_correct": 4287, + "selected_both_wrong": 1980 + }, + { + "budget": 0.65, + "threshold": -8.49548625946045, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.7289338674537915, + "rescued": 1285, + "harmed": 1589, + "missed_rescue": 84, + "avoided_harm": 1332, + "selected_both_correct": 4382, + "selected_both_wrong": 1992 + }, + { + "budget": 0.66, + "threshold": -8.5530366897583, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.7272471712699417, + "rescued": 1290, + "harmed": 1618, + "missed_rescue": 79, + "avoided_harm": 1303, + "selected_both_correct": 4487, + "selected_both_wrong": 1996 + }, + { + "budget": 0.67, + "threshold": -8.610180854797363, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.7263335441703563, + "rescued": 1294, + "harmed": 1635, + "missed_rescue": 75, + "avoided_harm": 1286, + "selected_both_correct": 4602, + "selected_both_wrong": 2002 + }, + { + "budget": 0.68, + "threshold": -8.674345016479492, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.724225173940544, + "rescued": 1296, + "harmed": 1667, + "missed_rescue": 73, + "avoided_harm": 1254, + "selected_both_correct": 4703, + "selected_both_wrong": 2009 + }, + { + "budget": 0.69, + "threshold": -8.73198127746582, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.7227493147796753, + "rescued": 1303, + "harmed": 1695, + "missed_rescue": 66, + "avoided_harm": 1226, + "selected_both_correct": 4805, + "selected_both_wrong": 2015 + }, + { + "budget": 0.7, + "threshold": -8.787148475646973, + "operator": "ge", + "actual_fraction": 0.6999789162977019, + "metric": 0.7210626185958254, + "rescued": 1304, + "harmed": 1720, + "missed_rescue": 65, + "avoided_harm": 1201, + "selected_both_correct": 4914, + "selected_both_wrong": 2022 + }, + { + "budget": 0.71, + "threshold": -8.846220016479492, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.7190948063813339, + "rescued": 1307, + "harmed": 1751, + "missed_rescue": 62, + "avoided_harm": 1170, + "selected_both_correct": 5009, + "selected_both_wrong": 2035 + }, + { + "budget": 0.72, + "threshold": -8.907856941223145, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.7169864361515216, + "rescued": 1310, + "harmed": 1784, + "missed_rescue": 59, + "avoided_harm": 1137, + "selected_both_correct": 5109, + "selected_both_wrong": 2041 + }, + { + "budget": 0.73, + "threshold": -8.959864616394043, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.71501862393703, + "rescued": 1315, + "harmed": 1817, + "missed_rescue": 54, + "avoided_harm": 1104, + "selected_both_correct": 5205, + "selected_both_wrong": 2050 + }, + { + "budget": 0.74, + "threshold": -9.021697998046875, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.7132616487455197, + "rescued": 1318, + "harmed": 1845, + "missed_rescue": 51, + "avoided_harm": 1076, + "selected_both_correct": 5313, + "selected_both_wrong": 2053 + }, + { + "budget": 0.75, + "threshold": -9.08212661743164, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.7112938365310282, + "rescued": 1323, + "harmed": 1878, + "missed_rescue": 46, + "avoided_harm": 1043, + "selected_both_correct": 5412, + "selected_both_wrong": 2058 + }, + { + "budget": 0.76, + "threshold": -9.147140502929688, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.7094665823318574, + "rescued": 1326, + "harmed": 1907, + "missed_rescue": 43, + "avoided_harm": 1014, + "selected_both_correct": 5518, + "selected_both_wrong": 2063 + }, + { + "budget": 0.77, + "threshold": -9.211226463317871, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.7072879330943848, + "rescued": 1328, + "harmed": 1940, + "missed_rescue": 41, + "avoided_harm": 981, + "selected_both_correct": 5623, + "selected_both_wrong": 2065 + }, + { + "budget": 0.78, + "threshold": -9.274300575256348, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.7053201208798932, + "rescued": 1331, + "harmed": 1971, + "missed_rescue": 38, + "avoided_harm": 950, + "selected_both_correct": 5721, + "selected_both_wrong": 2075 + }, + { + "budget": 0.79, + "threshold": -9.347284317016602, + "operator": "gt", + "actual_fraction": 0.7898657670953686, + "metric": 0.7028603556117787, + "rescued": 1331, + "harmed": 2006, + "missed_rescue": 38, + "avoided_harm": 915, + "selected_both_correct": 5822, + "selected_both_wrong": 2080 + }, + { + "budget": 0.8, + "threshold": -9.402959823608398, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.7004005903436643, + "rescued": 1334, + "harmed": 2044, + "missed_rescue": 35, + "avoided_harm": 877, + "selected_both_correct": 5924, + "selected_both_wrong": 2081 + }, + { + "budget": 0.81, + "threshold": -9.468785285949707, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.6978705460678896, + "rescued": 1337, + "harmed": 2083, + "missed_rescue": 32, + "avoided_harm": 838, + "selected_both_correct": 6021, + "selected_both_wrong": 2084 + }, + { + "budget": 0.82, + "threshold": -9.52985954284668, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.6950593857614731, + "rescued": 1340, + "harmed": 2126, + "missed_rescue": 29, + "avoided_harm": 795, + "selected_both_correct": 6112, + "selected_both_wrong": 2089 + }, + { + "budget": 0.83, + "threshold": -9.608570098876953, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.6924590624780378, + "rescued": 1341, + "harmed": 2164, + "missed_rescue": 28, + "avoided_harm": 757, + "selected_both_correct": 6210, + "selected_both_wrong": 2095 + }, + { + "budget": 0.84, + "threshold": -9.684678077697754, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.6905615292712067, + "rescued": 1341, + "harmed": 2191, + "missed_rescue": 28, + "avoided_harm": 730, + "selected_both_correct": 6320, + "selected_both_wrong": 2100 + }, + { + "budget": 0.85, + "threshold": -9.755184173583984, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.6873989739264882, + "rescued": 1343, + "harmed": 2238, + "missed_rescue": 26, + "avoided_harm": 683, + "selected_both_correct": 6407, + "selected_both_wrong": 2106 + }, + { + "budget": 0.86, + "threshold": -9.839400291442871, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.6847283716353925, + "rescued": 1343, + "harmed": 2276, + "missed_rescue": 26, + "avoided_harm": 645, + "selected_both_correct": 6508, + "selected_both_wrong": 2109 + }, + { + "budget": 0.87, + "threshold": -9.916629791259766, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.6819874903366364, + "rescued": 1346, + "harmed": 2318, + "missed_rescue": 23, + "avoided_harm": 603, + "selected_both_correct": 6602, + "selected_both_wrong": 2113 + }, + { + "budget": 0.88, + "threshold": -10.001229286193848, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.6798088410991637, + "rescued": 1349, + "harmed": 2352, + "missed_rescue": 20, + "avoided_harm": 569, + "selected_both_correct": 6702, + "selected_both_wrong": 2118 + }, + { + "budget": 0.89, + "threshold": -10.100711822509766, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.6767165647621055, + "rescued": 1350, + "harmed": 2397, + "missed_rescue": 19, + "avoided_harm": 524, + "selected_both_correct": 6795, + "selected_both_wrong": 2121 + }, + { + "budget": 0.9, + "threshold": -10.199607849121094, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.6744676365169724, + "rescued": 1351, + "harmed": 2430, + "missed_rescue": 18, + "avoided_harm": 491, + "selected_both_correct": 6899, + "selected_both_wrong": 2126 + }, + { + "budget": 0.91, + "threshold": -10.293088912963867, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.6717970342258768, + "rescued": 1352, + "harmed": 2469, + "missed_rescue": 17, + "avoided_harm": 452, + "selected_both_correct": 6997, + "selected_both_wrong": 2130 + }, + { + "budget": 0.92, + "threshold": -10.41290283203125, + "operator": "ge", + "actual_fraction": 0.9199522102747909, + "metric": 0.6689858739194603, + "rescued": 1356, + "harmed": 2513, + "missed_rescue": 13, + "avoided_harm": 408, + "selected_both_correct": 7083, + "selected_both_wrong": 2138 + }, + { + "budget": 0.93, + "threshold": -10.520768165588379, + "operator": "gt", + "actual_fraction": 0.929861550354909, + "metric": 0.6666666666666666, + "rescued": 1358, + "harmed": 2548, + "missed_rescue": 11, + "avoided_harm": 373, + "selected_both_correct": 7181, + "selected_both_wrong": 2144 + }, + { + "budget": 0.94, + "threshold": -10.65395736694336, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.663644669337269, + "rescued": 1362, + "harmed": 2595, + "missed_rescue": 7, + "avoided_harm": 326, + "selected_both_correct": 7272, + "selected_both_wrong": 2146 + }, + { + "budget": 0.95, + "threshold": -10.800084114074707, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.6603415559772297, + "rescued": 1364, + "harmed": 2644, + "missed_rescue": 5, + "avoided_harm": 277, + "selected_both_correct": 7355, + "selected_both_wrong": 2154 + }, + { + "budget": 0.96, + "threshold": -10.987924575805664, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.6574601166631527, + "rescued": 1365, + "harmed": 2686, + "missed_rescue": 4, + "avoided_harm": 235, + "selected_both_correct": 7449, + "selected_both_wrong": 2159 + }, + { + "budget": 0.97, + "threshold": -11.214167594909668, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.6540164452877926, + "rescued": 1365, + "harmed": 2735, + "missed_rescue": 4, + "avoided_harm": 186, + "selected_both_correct": 7538, + "selected_both_wrong": 2164 + }, + { + "budget": 0.98, + "threshold": -11.459996223449707, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.6504322158971115, + "rescued": 1368, + "harmed": 2789, + "missed_rescue": 1, + "avoided_harm": 132, + "selected_both_correct": 7617, + "selected_both_wrong": 2170 + }, + { + "budget": 0.99, + "threshold": -11.947364807128906, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.6460046384145056, + "rescued": 1368, + "harmed": 2852, + "missed_rescue": 1, + "avoided_harm": 69, + "selected_both_correct": 7688, + "selected_both_wrong": 2178 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.6412256658935975, + "rescued": 1369, + "harmed": 2921, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 7755, + "selected_both_wrong": 2184 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.6695119530385957, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2627, + "avoided_harm": 8602, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.430821418762207, + "operator": "ge", + "actual_fraction": 0.04055242038397187, + "metric": 0.67373734509259, + "rescued": 158, + "harmed": 9, + "missed_rescue": 2469, + "avoided_harm": 8593, + "selected_both_correct": 13, + "selected_both_wrong": 1250 + }, + { + "budget": 0.02, + "threshold": 0.8396652936935425, + "operator": "ge", + "actual_fraction": 0.05498681337379122, + "metric": 0.6756089952641579, + "rescued": 250, + "harmed": 35, + "missed_rescue": 2377, + "avoided_harm": 8567, + "selected_both_correct": 54, + "selected_both_wrong": 1600 + }, + { + "budget": 0.03, + "threshold": -0.16024979948997498, + "operator": "ge", + "actual_fraction": 0.0680600062388339, + "metric": 0.6773104954201288, + "rescued": 339, + "harmed": 64, + "missed_rescue": 2288, + "avoided_harm": 8538, + "selected_both_correct": 90, + "selected_both_wrong": 1907 + }, + { + "budget": 0.04, + "threshold": -0.9098284244537354, + "operator": "ge", + "actual_fraction": 0.08110484076794373, + "metric": 0.6788134872245697, + "rescued": 431, + "harmed": 103, + "missed_rescue": 2196, + "avoided_harm": 8499, + "selected_both_correct": 146, + "selected_both_wrong": 2180 + }, + { + "budget": 0.05, + "threshold": -1.4125885963439941, + "operator": "ge", + "actual_fraction": 0.09125712503190313, + "metric": 0.6796075206306894, + "rescued": 498, + "harmed": 142, + "missed_rescue": 2129, + "avoided_harm": 8460, + "selected_both_correct": 194, + "selected_both_wrong": 2384 + }, + { + "budget": 0.06, + "threshold": -1.8043409585952759, + "operator": "ge", + "actual_fraction": 0.10106910926466835, + "metric": 0.6802597623571449, + "rescued": 561, + "harmed": 182, + "missed_rescue": 2066, + "avoided_harm": 8420, + "selected_both_correct": 234, + "selected_both_wrong": 2587 + }, + { + "budget": 0.07, + "threshold": -2.2280092239379883, + "operator": "ge", + "actual_fraction": 0.11527663556702493, + "metric": 0.6812523041147945, + "rescued": 655, + "harmed": 241, + "missed_rescue": 1972, + "avoided_harm": 8361, + "selected_both_correct": 306, + "selected_both_wrong": 2863 + }, + { + "budget": 0.08, + "threshold": -2.5369863510131836, + "operator": "ge", + "actual_fraction": 0.12605280322150697, + "metric": 0.681167229106996, + "rescued": 713, + "harmed": 302, + "missed_rescue": 1914, + "avoided_harm": 8300, + "selected_both_correct": 378, + "selected_both_wrong": 3052 + }, + { + "budget": 0.09, + "threshold": -2.807616949081421, + "operator": "ge", + "actual_fraction": 0.13674389586819047, + "metric": 0.6818194708334515, + "rescued": 784, + "harmed": 350, + "missed_rescue": 1843, + "avoided_harm": 8252, + "selected_both_correct": 444, + "selected_both_wrong": 3244 + }, + { + "budget": 0.1, + "threshold": -3.0437393188476562, + "operator": "ge", + "actual_fraction": 0.14868275529591923, + "metric": 0.681535887474123, + "rescued": 843, + "harmed": 419, + "missed_rescue": 1784, + "avoided_harm": 8183, + "selected_both_correct": 513, + "selected_both_wrong": 3468 + }, + { + "budget": 0.11, + "threshold": -3.299111843109131, + "operator": "ge", + "actual_fraction": 0.1613305731219692, + "metric": 0.6816493208178545, + "rescued": 910, + "harmed": 482, + "missed_rescue": 1717, + "avoided_harm": 8120, + "selected_both_correct": 594, + "selected_both_wrong": 3703 + }, + { + "budget": 0.12, + "threshold": -3.5267717838287354, + "operator": "ge", + "actual_fraction": 0.17383659926835493, + "metric": 0.6807985707398689, + "rescued": 966, + "harmed": 568, + "missed_rescue": 1661, + "avoided_harm": 8034, + "selected_both_correct": 669, + "selected_both_wrong": 3927 + }, + { + "budget": 0.13, + "threshold": -3.745974540710449, + "operator": "ge", + "actual_fraction": 0.18634262541474067, + "metric": 0.6802881206930778, + "rescued": 1037, + "harmed": 657, + "missed_rescue": 1590, + "avoided_harm": 7945, + "selected_both_correct": 761, + "selected_both_wrong": 4116 + }, + { + "budget": 0.14, + "threshold": -3.9486825466156006, + "operator": "ge", + "actual_fraction": 0.19865014320959645, + "metric": 0.6797209539744208, + "rescued": 1108, + "harmed": 748, + "missed_rescue": 1519, + "avoided_harm": 7854, + "selected_both_correct": 842, + "selected_both_wrong": 4307 + }, + { + "budget": 0.15, + "threshold": -4.13905668258667, + "operator": "ge", + "actual_fraction": 0.21149646938717637, + "metric": 0.6788418455605025, + "rescued": 1175, + "harmed": 846, + "missed_rescue": 1452, + "avoided_harm": 7756, + "selected_both_correct": 934, + "selected_both_wrong": 4503 + }, + { + "budget": 0.16, + "threshold": -4.306515693664551, + "operator": "ge", + "actual_fraction": 0.2222726370416584, + "metric": 0.6784448288574426, + "rescued": 1233, + "harmed": 918, + "missed_rescue": 1394, + "avoided_harm": 7684, + "selected_both_correct": 1022, + "selected_both_wrong": 4665 + }, + { + "budget": 0.17, + "threshold": -4.4712700843811035, + "operator": "ge", + "actual_fraction": 0.23472194651617842, + "metric": 0.6770836287326659, + "rescued": 1283, + "harmed": 1016, + "missed_rescue": 1344, + "avoided_harm": 7586, + "selected_both_correct": 1129, + "selected_both_wrong": 4849 + }, + { + "budget": 0.18, + "threshold": -4.667325019836426, + "operator": "ge", + "actual_fraction": 0.24966678955278904, + "metric": 0.6750701868814338, + "rescued": 1337, + "harmed": 1141, + "missed_rescue": 1290, + "avoided_harm": 7461, + "selected_both_correct": 1257, + "selected_both_wrong": 5069 + }, + { + "budget": 0.19, + "threshold": -4.808548450469971, + "operator": "gt", + "actual_fraction": 0.2611235572696594, + "metric": 0.6740492867878513, + "rescued": 1394, + "harmed": 1234, + "missed_rescue": 1233, + "avoided_harm": 7368, + "selected_both_correct": 1363, + "selected_both_wrong": 5217 + }, + { + "budget": 0.2, + "threshold": -4.945236682891846, + "operator": "ge", + "actual_fraction": 0.27226838329126846, + "metric": 0.6728298783427389, + "rescued": 1441, + "harmed": 1324, + "missed_rescue": 1186, + "avoided_harm": 7278, + "selected_both_correct": 1462, + "selected_both_wrong": 5374 + }, + { + "budget": 0.21, + "threshold": -5.0980048179626465, + "operator": "ge", + "actual_fraction": 0.28545500950004254, + "metric": 0.6707030031477753, + "rescued": 1484, + "harmed": 1442, + "missed_rescue": 1143, + "avoided_harm": 7160, + "selected_both_correct": 1602, + "selected_both_wrong": 5538 + }, + { + "budget": 0.22, + "threshold": -5.24181604385376, + "operator": "ge", + "actual_fraction": 0.29620281881859173, + "metric": 0.6688313529762073, + "rescued": 1521, + "harmed": 1545, + "missed_rescue": 1106, + "avoided_harm": 7057, + "selected_both_correct": 1712, + "selected_both_wrong": 5667 + }, + { + "budget": 0.23, + "threshold": -5.381783485412598, + "operator": "ge", + "actual_fraction": 0.30904914499617164, + "metric": 0.6671865694921022, + "rescued": 1570, + "harmed": 1652, + "missed_rescue": 1057, + "avoided_harm": 6950, + "selected_both_correct": 1879, + "selected_both_wrong": 5797 + }, + { + "budget": 0.24, + "threshold": -5.480440616607666, + "operator": "ge", + "actual_fraction": 0.31880441255707115, + "metric": 0.6657119360235941, + "rescued": 1608, + "harmed": 1742, + "missed_rescue": 1019, + "avoided_harm": 6860, + "selected_both_correct": 1986, + "selected_both_wrong": 5906 + }, + { + "budget": 0.25, + "threshold": -5.592668533325195, + "operator": "ge", + "actual_fraction": 0.3278223633837166, + "metric": 0.6640104358676233, + "rescued": 1630, + "harmed": 1824, + "missed_rescue": 997, + "avoided_harm": 6778, + "selected_both_correct": 2094, + "selected_both_wrong": 6012 + }, + { + "budget": 0.26, + "threshold": -5.701513290405273, + "operator": "ge", + "actual_fraction": 0.3389388310693928, + "metric": 0.6622805773757195, + "rescued": 1667, + "harmed": 1922, + "missed_rescue": 960, + "avoided_harm": 6680, + "selected_both_correct": 2235, + "selected_both_wrong": 6128 + }, + { + "budget": 0.27, + "threshold": -5.817866802215576, + "operator": "ge", + "actual_fraction": 0.35082097382525596, + "metric": 0.6600402688370246, + "rescued": 1699, + "harmed": 2033, + "missed_rescue": 928, + "avoided_harm": 6569, + "selected_both_correct": 2400, + "selected_both_wrong": 6239 + }, + { + "budget": 0.28, + "threshold": -5.915761947631836, + "operator": "ge", + "actual_fraction": 0.36142699146414087, + "metric": 0.658111901993591, + "rescued": 1731, + "harmed": 2133, + "missed_rescue": 896, + "avoided_harm": 6469, + "selected_both_correct": 2541, + "selected_both_wrong": 6340 + }, + { + "budget": 0.29, + "threshold": -6.032350063323975, + "operator": "ge", + "actual_fraction": 0.37316734254033973, + "metric": 0.6560133851345603, + "rescued": 1770, + "harmed": 2246, + "missed_rescue": 857, + "avoided_harm": 6356, + "selected_both_correct": 2702, + "selected_both_wrong": 6441 + }, + { + "budget": 0.3, + "threshold": -6.134230136871338, + "operator": "ge", + "actual_fraction": 0.38351813515582905, + "metric": 0.6541417349629923, + "rescued": 1793, + "harmed": 2335, + "missed_rescue": 834, + "avoided_harm": 6267, + "selected_both_correct": 2873, + "selected_both_wrong": 6523 + }, + { + "budget": 0.31, + "threshold": -6.218080520629883, + "operator": "ge", + "actual_fraction": 0.39270623599807164, + "metric": 0.6522417264554916, + "rescued": 1815, + "harmed": 2424, + "missed_rescue": 812, + "avoided_harm": 6178, + "selected_both_correct": 3002, + "selected_both_wrong": 6607 + }, + { + "budget": 0.32, + "threshold": -6.302128791809082, + "operator": "ge", + "actual_fraction": 0.4026032952386354, + "metric": 0.6502282846042594, + "rescued": 1845, + "harmed": 2525, + "missed_rescue": 782, + "avoided_harm": 6077, + "selected_both_correct": 3150, + "selected_both_wrong": 6677 + }, + { + "budget": 0.33, + "threshold": -6.377316474914551, + "operator": "ge", + "actual_fraction": 0.4103167626123699, + "metric": 0.6486685761279528, + "rescued": 1864, + "harmed": 2599, + "missed_rescue": 763, + "avoided_harm": 6003, + "selected_both_correct": 3271, + "selected_both_wrong": 6735 + }, + { + "budget": 0.34, + "threshold": -6.449829578399658, + "operator": "ge", + "actual_fraction": 0.4194481467827468, + "metric": 0.6470805093157134, + "rescued": 1889, + "harmed": 2680, + "missed_rescue": 738, + "avoided_harm": 5922, + "selected_both_correct": 3412, + "selected_both_wrong": 6810 + }, + { + "budget": 0.35, + "threshold": -6.539102077484131, + "operator": "ge", + "actual_fraction": 0.4308198394918186, + "metric": 0.6448685591129513, + "rescued": 1916, + "harmed": 2785, + "missed_rescue": 711, + "avoided_harm": 5817, + "selected_both_correct": 3593, + "selected_both_wrong": 6898 + }, + { + "budget": 0.36, + "threshold": -6.614948272705078, + "operator": "ge", + "actual_fraction": 0.4392139069279415, + "metric": 0.6431387006210476, + "rescued": 1935, + "harmed": 2865, + "missed_rescue": 692, + "avoided_harm": 5737, + "selected_both_correct": 3729, + "selected_both_wrong": 6959 + }, + { + "budget": 0.37, + "threshold": -6.686672210693359, + "operator": "ge", + "actual_fraction": 0.4474661826844001, + "metric": 0.6415222754728752, + "rescued": 1954, + "harmed": 2941, + "missed_rescue": 673, + "avoided_harm": 5661, + "selected_both_correct": 3881, + "selected_both_wrong": 7003 + }, + { + "budget": 0.38, + "threshold": -6.78654146194458, + "operator": "ge", + "actual_fraction": 0.46124833394776393, + "metric": 0.6382043501687321, + "rescued": 1979, + "harmed": 3083, + "missed_rescue": 648, + "avoided_harm": 5519, + "selected_both_correct": 4094, + "selected_both_wrong": 7109 + }, + { + "budget": 0.39, + "threshold": -6.846216201782227, + "operator": "ge", + "actual_fraction": 0.46918866800896125, + "metric": 0.6371550917392167, + "rescued": 1999, + "harmed": 3140, + "missed_rescue": 628, + "avoided_harm": 5462, + "selected_both_correct": 4243, + "selected_both_wrong": 7163 + }, + { + "budget": 0.4, + "threshold": -6.915761470794678, + "operator": "ge", + "actual_fraction": 0.47786631880441255, + "metric": 0.6349714998723874, + "rescued": 2015, + "harmed": 3233, + "missed_rescue": 612, + "avoided_harm": 5369, + "selected_both_correct": 4390, + "selected_both_wrong": 7213 + }, + { + "budget": 0.41, + "threshold": -6.9817047119140625, + "operator": "ge", + "actual_fraction": 0.48657232793579674, + "metric": 0.6325043246462297, + "rescued": 2033, + "harmed": 3338, + "missed_rescue": 594, + "avoided_harm": 5264, + "selected_both_correct": 4523, + "selected_both_wrong": 7264 + }, + { + "budget": 0.42, + "threshold": -7.047516822814941, + "operator": "gt", + "actual_fraction": 0.4953634120749794, + "metric": 0.6308878994980575, + "rescued": 2055, + "harmed": 3417, + "missed_rescue": 572, + "avoided_harm": 5185, + "selected_both_correct": 4686, + "selected_both_wrong": 7310 + }, + { + "budget": 0.43, + "threshold": -7.1213059425354, + "operator": "ge", + "actual_fraction": 0.5059694297138644, + "metric": 0.6287610243030939, + "rescued": 2086, + "harmed": 3523, + "missed_rescue": 541, + "avoided_harm": 5079, + "selected_both_correct": 4852, + "selected_both_wrong": 7381 + }, + { + "budget": 0.44, + "threshold": -7.181412696838379, + "operator": "ge", + "actual_fraction": 0.51439185548592, + "metric": 0.6270311658111902, + "rescued": 2107, + "harmed": 3605, + "missed_rescue": 520, + "avoided_harm": 4997, + "selected_both_correct": 5002, + "selected_both_wrong": 7425 + }, + { + "budget": 0.45, + "threshold": -7.25251579284668, + "operator": "ge", + "actual_fraction": 0.524629214757678, + "metric": 0.6243087655616368, + "rescued": 2122, + "harmed": 3716, + "missed_rescue": 505, + "avoided_harm": 4886, + "selected_both_correct": 5184, + "selected_both_wrong": 7478 + }, + { + "budget": 0.46, + "threshold": -7.320802211761475, + "operator": "ge", + "actual_fraction": 0.5349232907013016, + "metric": 0.6222669653744718, + "rescued": 2145, + "harmed": 3811, + "missed_rescue": 482, + "avoided_harm": 4791, + "selected_both_correct": 5373, + "selected_both_wrong": 7534 + }, + { + "budget": 0.47, + "threshold": -7.374159812927246, + "operator": "ge", + "actual_fraction": 0.5420412330204464, + "metric": 0.620934123585628, + "rescued": 2158, + "harmed": 3871, + "missed_rescue": 469, + "avoided_harm": 4731, + "selected_both_correct": 5508, + "selected_both_wrong": 7577 + }, + { + "budget": 0.48, + "threshold": -7.4538164138793945, + "operator": "ge", + "actual_fraction": 0.5530159090264584, + "metric": 0.6183818733516717, + "rescued": 2178, + "harmed": 3981, + "missed_rescue": 449, + "avoided_harm": 4621, + "selected_both_correct": 5711, + "selected_both_wrong": 7631 + }, + { + "budget": 0.49, + "threshold": -7.532625675201416, + "operator": "ge", + "actual_fraction": 0.5640189433684032, + "metric": 0.6162549981567081, + "rescued": 2202, + "harmed": 4080, + "missed_rescue": 425, + "avoided_harm": 4522, + "selected_both_correct": 5928, + "selected_both_wrong": 7679 + }, + { + "budget": 0.5, + "threshold": -7.600386619567871, + "operator": "ge", + "actual_fraction": 0.5738876442730341, + "metric": 0.6134475228993562, + "rescued": 2214, + "harmed": 4191, + "missed_rescue": 413, + "avoided_harm": 4411, + "selected_both_correct": 6117, + "selected_both_wrong": 7715 + }, + { + "budget": 0.51, + "threshold": -7.662148952484131, + "operator": "ge", + "actual_fraction": 0.5832458951308738, + "metric": 0.6109519893372657, + "rescued": 2221, + "harmed": 4286, + "missed_rescue": 406, + "avoided_harm": 4316, + "selected_both_correct": 6292, + "selected_both_wrong": 7768 + }, + { + "budget": 0.52, + "threshold": -7.733202934265137, + "operator": "ge", + "actual_fraction": 0.5937101210900945, + "metric": 0.6086549641267051, + "rescued": 2242, + "harmed": 4388, + "missed_rescue": 385, + "avoided_harm": 4214, + "selected_both_correct": 6483, + "selected_both_wrong": 7823 + }, + { + "budget": 0.53, + "threshold": -7.79387092590332, + "operator": "ge", + "actual_fraction": 0.6031250886197997, + "metric": 0.6062728639083459, + "rescued": 2260, + "harmed": 4490, + "missed_rescue": 367, + "avoided_harm": 4112, + "selected_both_correct": 6652, + "selected_both_wrong": 7866 + }, + { + "budget": 0.54, + "threshold": -7.855954647064209, + "operator": "ge", + "actual_fraction": 0.611263931032527, + "metric": 0.6044579304086436, + "rescued": 2271, + "harmed": 4565, + "missed_rescue": 356, + "avoided_harm": 4037, + "selected_both_correct": 6810, + "selected_both_wrong": 7909 + }, + { + "budget": 0.55, + "threshold": -7.925965785980225, + "operator": "ge", + "actual_fraction": 0.6218983070073448, + "metric": 0.6018489635028217, + "rescued": 2283, + "harmed": 4669, + "missed_rescue": 344, + "avoided_harm": 3933, + "selected_both_correct": 7013, + "selected_both_wrong": 7965 + }, + { + "budget": 0.56, + "threshold": -7.977593421936035, + "operator": "ge", + "actual_fraction": 0.6296117743810793, + "metric": 0.5999489549953209, + "rescued": 2291, + "harmed": 4744, + "missed_rescue": 336, + "avoided_harm": 3858, + "selected_both_correct": 7173, + "selected_both_wrong": 7994 + }, + { + "budget": 0.57, + "threshold": -8.031231880187988, + "operator": "ge", + "actual_fraction": 0.6381192751609336, + "metric": 0.5979638714800215, + "rescued": 2306, + "harmed": 4829, + "missed_rescue": 321, + "avoided_harm": 3773, + "selected_both_correct": 7339, + "selected_both_wrong": 8028 + }, + { + "budget": 0.58, + "threshold": -8.088851928710938, + "operator": "ge", + "actual_fraction": 0.6469387176360492, + "metric": 0.5955534129257295, + "rescued": 2318, + "harmed": 4926, + "missed_rescue": 309, + "avoided_harm": 3676, + "selected_both_correct": 7520, + "selected_both_wrong": 8049 + }, + { + "budget": 0.59, + "threshold": -8.145581245422363, + "operator": "ge", + "actual_fraction": 0.654935768369112, + "metric": 0.5933414627229674, + "rescued": 2329, + "harmed": 5015, + "missed_rescue": 298, + "avoided_harm": 3587, + "selected_both_correct": 7677, + "selected_both_wrong": 8074 + }, + { + "budget": 0.6, + "threshold": -8.199271202087402, + "operator": "ge", + "actual_fraction": 0.6636701358364291, + "metric": 0.5907608541530783, + "rescued": 2339, + "harmed": 5116, + "missed_rescue": 288, + "avoided_harm": 3486, + "selected_both_correct": 7851, + "selected_both_wrong": 8097 + }, + { + "budget": 0.61, + "threshold": -8.256582260131836, + "operator": "ge", + "actual_fraction": 0.6710149448430366, + "metric": 0.5888324873096447, + "rescued": 2347, + "harmed": 5192, + "missed_rescue": 280, + "avoided_harm": 3410, + "selected_both_correct": 8006, + "selected_both_wrong": 8117 + }, + { + "budget": 0.62, + "threshold": -8.316590309143066, + "operator": "ge", + "actual_fraction": 0.6805433457164733, + "metric": 0.5859966537163599, + "rescued": 2355, + "harmed": 5300, + "missed_rescue": 272, + "avoided_harm": 3302, + "selected_both_correct": 8192, + "selected_both_wrong": 8151 + }, + { + "budget": 0.63, + "threshold": -8.380763053894043, + "operator": "ge", + "actual_fraction": 0.6900150299180444, + "metric": 0.5836712701698664, + "rescued": 2370, + "harmed": 5397, + "missed_rescue": 257, + "avoided_harm": 3205, + "selected_both_correct": 8387, + "selected_both_wrong": 8178 + }, + { + "budget": 0.64, + "threshold": -8.436734199523926, + "operator": "gt", + "actual_fraction": 0.6979837223151746, + "metric": 0.5813458866233729, + "rescued": 2380, + "harmed": 5489, + "missed_rescue": 247, + "avoided_harm": 3113, + "selected_both_correct": 8531, + "selected_both_wrong": 8213 + }, + { + "budget": 0.65, + "threshold": -8.49548625946045, + "operator": "ge", + "actual_fraction": 0.7061225647279018, + "metric": 0.579105578084678, + "rescued": 2387, + "harmed": 5575, + "missed_rescue": 240, + "avoided_harm": 3027, + "selected_both_correct": 8704, + "selected_both_wrong": 8234 + }, + { + "budget": 0.66, + "threshold": -8.5530366897583, + "operator": "ge", + "actual_fraction": 0.7130703570314494, + "metric": 0.5778578113036327, + "rescued": 2398, + "harmed": 5630, + "missed_rescue": 229, + "avoided_harm": 2972, + "selected_both_correct": 8865, + "selected_both_wrong": 8252 + }, + { + "budget": 0.67, + "threshold": -8.610180854797363, + "operator": "ge", + "actual_fraction": 0.7218614411706321, + "metric": 0.5755891444290049, + "rescued": 2408, + "harmed": 5720, + "missed_rescue": 219, + "avoided_harm": 2882, + "selected_both_correct": 9038, + "selected_both_wrong": 8289 + }, + { + "budget": 0.68, + "threshold": -8.674345016479492, + "operator": "ge", + "actual_fraction": 0.7307092419816805, + "metric": 0.5725548024841902, + "rescued": 2415, + "harmed": 5834, + "missed_rescue": 212, + "avoided_harm": 2768, + "selected_both_correct": 9193, + "selected_both_wrong": 8325 + }, + { + "budget": 0.69, + "threshold": -8.73198127746582, + "operator": "ge", + "actual_fraction": 0.7391033094178033, + "metric": 0.5702861356095624, + "rescued": 2426, + "harmed": 5925, + "missed_rescue": 201, + "avoided_harm": 2677, + "selected_both_correct": 9357, + "selected_both_wrong": 8355 + }, + { + "budget": 0.7, + "threshold": -8.787148475646973, + "operator": "ge", + "actual_fraction": 0.7464481184244108, + "metric": 0.5685279187817259, + "rescued": 2432, + "harmed": 5993, + "missed_rescue": 195, + "avoided_harm": 2609, + "selected_both_correct": 9512, + "selected_both_wrong": 8385 + }, + { + "budget": 0.71, + "threshold": -8.846220016479492, + "operator": "ge", + "actual_fraction": 0.7537078524232198, + "metric": 0.5663726852508295, + "rescued": 2438, + "harmed": 6075, + "missed_rescue": 189, + "avoided_harm": 2527, + "selected_both_correct": 9662, + "selected_both_wrong": 8403 + }, + { + "budget": 0.72, + "threshold": -8.907856941223145, + "operator": "ge", + "actual_fraction": 0.7623855032186712, + "metric": 0.5639622266965374, + "rescued": 2445, + "harmed": 6167, + "missed_rescue": 182, + "avoided_harm": 2435, + "selected_both_correct": 9844, + "selected_both_wrong": 8428 + }, + { + "budget": 0.73, + "threshold": -8.959864616394043, + "operator": "ge", + "actual_fraction": 0.7701556872642713, + "metric": 0.5618637098375068, + "rescued": 2455, + "harmed": 6251, + "missed_rescue": 172, + "avoided_harm": 2351, + "selected_both_correct": 10001, + "selected_both_wrong": 8451 + }, + { + "budget": 0.74, + "threshold": -9.021697998046875, + "operator": "ge", + "actual_fraction": 0.7780393046536029, + "metric": 0.5599069846581403, + "rescued": 2463, + "harmed": 6328, + "missed_rescue": 164, + "avoided_harm": 2274, + "selected_both_correct": 10172, + "selected_both_wrong": 8473 + }, + { + "budget": 0.75, + "threshold": -9.08212661743164, + "operator": "ge", + "actual_fraction": 0.7862348637381958, + "metric": 0.5578651844709752, + "rescued": 2472, + "harmed": 6409, + "missed_rescue": 155, + "avoided_harm": 2193, + "selected_both_correct": 10346, + "selected_both_wrong": 8498 + }, + { + "budget": 0.76, + "threshold": -9.147140502929688, + "operator": "ge", + "actual_fraction": 0.7948557978617815, + "metric": 0.5555965175963474, + "rescued": 2478, + "harmed": 6495, + "missed_rescue": 149, + "avoided_harm": 2107, + "selected_both_correct": 10525, + "selected_both_wrong": 8531 + }, + { + "budget": 0.77, + "threshold": -9.211226463317871, + "operator": "ge", + "actual_fraction": 0.8029662819385759, + "metric": 0.5536681507529139, + "rescued": 2489, + "harmed": 6574, + "missed_rescue": 138, + "avoided_harm": 2028, + "selected_both_correct": 10712, + "selected_both_wrong": 8540 + }, + { + "budget": 0.78, + "threshold": -9.274300575256348, + "operator": "ge", + "actual_fraction": 0.8114737827184301, + "metric": 0.5513144088704874, + "rescued": 2498, + "harmed": 6666, + "missed_rescue": 129, + "avoided_harm": 1936, + "selected_both_correct": 10890, + "selected_both_wrong": 8561 + }, + { + "budget": 0.79, + "threshold": -9.347284317016602, + "operator": "gt", + "actual_fraction": 0.8210872585996654, + "metric": 0.5488188753083969, + "rescued": 2507, + "harmed": 6763, + "missed_rescue": 120, + "avoided_harm": 1839, + "selected_both_correct": 11095, + "selected_both_wrong": 8589 + }, + { + "budget": 0.8, + "threshold": -9.402959823608398, + "operator": "ge", + "actual_fraction": 0.8279499758954144, + "metric": 0.5472591668320903, + "rescued": 2518, + "harmed": 6829, + "missed_rescue": 109, + "avoided_harm": 1773, + "selected_both_correct": 11239, + "selected_both_wrong": 8610 + }, + { + "budget": 0.81, + "threshold": -9.468785285949707, + "operator": "ge", + "actual_fraction": 0.8352664265660891, + "metric": 0.5455293083401866, + "rescued": 2521, + "harmed": 6893, + "missed_rescue": 106, + "avoided_harm": 1709, + "selected_both_correct": 11407, + "selected_both_wrong": 8633 + }, + { + "budget": 0.82, + "threshold": -9.52985954284668, + "operator": "ge", + "actual_fraction": 0.8424978022289652, + "metric": 0.5440546748716786, + "rescued": 2526, + "harmed": 6950, + "missed_rescue": 101, + "avoided_harm": 1652, + "selected_both_correct": 11577, + "selected_both_wrong": 8656 + }, + { + "budget": 0.83, + "threshold": -9.608570098876953, + "operator": "ge", + "actual_fraction": 0.850891869665088, + "metric": 0.5418143663329836, + "rescued": 2528, + "harmed": 7031, + "missed_rescue": 99, + "avoided_harm": 1571, + "selected_both_correct": 11770, + "selected_both_wrong": 8676 + }, + { + "budget": 0.84, + "threshold": -9.684678077697754, + "operator": "ge", + "actual_fraction": 0.8593710121090095, + "metric": 0.53948898278649, + "rescued": 2534, + "harmed": 7119, + "missed_rescue": 93, + "avoided_harm": 1483, + "selected_both_correct": 11952, + "selected_both_wrong": 8699 + }, + { + "budget": 0.85, + "threshold": -9.755184173583984, + "operator": "ge", + "actual_fraction": 0.8669994044749454, + "metric": 0.5374188242633923, + "rescued": 2538, + "harmed": 7196, + "missed_rescue": 89, + "avoided_harm": 1406, + "selected_both_correct": 12113, + "selected_both_wrong": 8726 + }, + { + "budget": 0.86, + "threshold": -9.839400291442871, + "operator": "ge", + "actual_fraction": 0.877293480418569, + "metric": 0.5350934407168987, + "rescued": 2549, + "harmed": 7289, + "missed_rescue": 78, + "avoided_harm": 1313, + "selected_both_correct": 12347, + "selected_both_wrong": 8751 + }, + { + "budget": 0.87, + "threshold": -9.916629791259766, + "operator": "ge", + "actual_fraction": 0.8848651561126393, + "metric": 0.5330799988656666, + "rescued": 2555, + "harmed": 7366, + "missed_rescue": 72, + "avoided_harm": 1236, + "selected_both_correct": 12510, + "selected_both_wrong": 8772 + }, + { + "budget": 0.88, + "threshold": -10.001229286193848, + "operator": "ge", + "actual_fraction": 0.893089073533165, + "metric": 0.5305561069676431, + "rescued": 2559, + "harmed": 7459, + "missed_rescue": 68, + "avoided_harm": 1143, + "selected_both_correct": 12685, + "selected_both_wrong": 8790 + }, + { + "budget": 0.89, + "threshold": -10.100711822509766, + "operator": "ge", + "actual_fraction": 0.901653290984885, + "metric": 0.5283725151008138, + "rescued": 2567, + "harmed": 7544, + "missed_rescue": 60, + "avoided_harm": 1058, + "selected_both_correct": 12873, + "selected_both_wrong": 8811 + }, + { + "budget": 0.9, + "threshold": -10.199607849121094, + "operator": "ge", + "actual_fraction": 0.9106996001474633, + "metric": 0.5261889232339846, + "rescued": 2572, + "harmed": 7626, + "missed_rescue": 55, + "avoided_harm": 976, + "selected_both_correct": 13080, + "selected_both_wrong": 8836 + }, + { + "budget": 0.91, + "threshold": -10.293088912963867, + "operator": "ge", + "actual_fraction": 0.9177324674588095, + "metric": 0.5241754813827525, + "rescued": 2575, + "harmed": 7700, + "missed_rescue": 52, + "avoided_harm": 902, + "selected_both_correct": 13240, + "selected_both_wrong": 8847 + }, + { + "budget": 0.92, + "threshold": -10.41290283203125, + "operator": "ge", + "actual_fraction": 0.9268638516291864, + "metric": 0.5215948728128633, + "rescued": 2580, + "harmed": 7796, + "missed_rescue": 47, + "avoided_harm": 806, + "selected_both_correct": 13438, + "selected_both_wrong": 8870 + }, + { + "budget": 0.93, + "threshold": -10.520768165588379, + "operator": "gt", + "actual_fraction": 0.9340668689561297, + "metric": 0.5198650143209597, + "rescued": 2589, + "harmed": 7866, + "missed_rescue": 38, + "avoided_harm": 736, + "selected_both_correct": 13595, + "selected_both_wrong": 8888 + }, + { + "budget": 0.94, + "threshold": -10.65395736694336, + "operator": "ge", + "actual_fraction": 0.9422907863766554, + "metric": 0.5175963474463319, + "rescued": 2594, + "harmed": 7951, + "missed_rescue": 33, + "avoided_harm": 651, + "selected_both_correct": 13777, + "selected_both_wrong": 8906 + }, + { + "budget": 0.95, + "threshold": -10.800084114074707, + "operator": "ge", + "actual_fraction": 0.9508266454924426, + "metric": 0.5152426055639056, + "rescued": 2600, + "harmed": 8040, + "missed_rescue": 27, + "avoided_harm": 562, + "selected_both_correct": 13959, + "selected_both_wrong": 8930 + }, + { + "budget": 0.96, + "threshold": -10.987924575805664, + "operator": "ge", + "actual_fraction": 0.9601281796784165, + "metric": 0.5127187136658821, + "rescued": 2612, + "harmed": 8141, + "missed_rescue": 15, + "avoided_harm": 461, + "selected_both_correct": 14155, + "selected_both_wrong": 8949 + }, + { + "budget": 0.97, + "threshold": -11.214167594909668, + "operator": "ge", + "actual_fraction": 0.9683237387630094, + "metric": 0.509939596744463, + "rescued": 2614, + "harmed": 8241, + "missed_rescue": 13, + "avoided_harm": 361, + "selected_both_correct": 14328, + "selected_both_wrong": 8963 + }, + { + "budget": 0.98, + "threshold": -11.459996223449707, + "operator": "ge", + "actual_fraction": 0.976944672886595, + "metric": 0.506791821455917, + "rescued": 2617, + "harmed": 8355, + "missed_rescue": 10, + "avoided_harm": 247, + "selected_both_correct": 14505, + "selected_both_wrong": 8973 + }, + { + "budget": 0.99, + "threshold": -11.947364807128906, + "operator": "ge", + "actual_fraction": 0.9883163655956668, + "metric": 0.5033604628080425, + "rescued": 2622, + "harmed": 8481, + "missed_rescue": 5, + "avoided_harm": 121, + "selected_both_correct": 14753, + "selected_both_wrong": 8995 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5000708958398321, + "rescued": 2627, + "harmed": 8602, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 15007, + "selected_both_wrong": 9027 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 7755, + "rescuable": 1369, + "harmful": 2921, + "both_wrong": 2184, + "unit": "sample", + "total": 14229 + }, + "test": { + "both_correct": 15007, + "rescuable": 2627, + "harmful": 8602, + "both_wrong": 9027, + "unit": "sample", + "total": 35263 + } + }, + "small": { + "validation": 0.7502986857825568, + "test": 0.6695119530385957 + }, + "large": { + "validation": 0.6412256658935975, + "test": 0.5000708958398321 + }, + "learned_minus_confidence_test": 0.048407679437370565, + "learned_minus_small_test": -0.002325383546493476, + "learned_minus_large_test": 0.1671156736522701 + }, + { + "key": [ + "grefcoco", + "instancevg", + "Qwen3.8" + ], + "expert_display": "InstanceVG", + "run": "shared_12_instancevg_Qwen3.8", + "validation_rows": 14229, + "test_rows": 35263, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detections_above_07", + "max_score", + "mean_score", + "query_count", + "detections_above_07_valid", + "max_score_valid", + "mean_score_valid", + "query_count_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/instancevg/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 71, + "validation_area_0_25": 0.7477616136060159, + "validation_point": { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.8201560193970061, + "rescued": 1660, + "harmed": 666, + "missed_rescue": 304, + "avoided_harm": 505, + "selected_both_correct": 6505, + "selected_both_wrong": 1271 + }, + "test_point": { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.6838612710206166, + "metric": 0.7243286163967898, + "rescued": 3794, + "harmed": 1861, + "missed_rescue": 1629, + "avoided_harm": 910, + "selected_both_correct": 13678, + "selected_both_wrong": 4782 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7502986857825568, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1964, + "avoided_harm": 1171, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -2.0005727208172175e-07, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.7502284067748963, + "rescued": 0, + "harmed": 1, + "missed_rescue": 1964, + "avoided_harm": 1170, + "selected_both_correct": 141, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -2.4156264544217265e-07, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.7501581277672359, + "rescued": 0, + "harmed": 2, + "missed_rescue": 1964, + "avoided_harm": 1169, + "selected_both_correct": 282, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -2.7729177531909954e-07, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.7498770117365943, + "rescued": 0, + "harmed": 6, + "missed_rescue": 1964, + "avoided_harm": 1165, + "selected_both_correct": 420, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": -3.064858447032748e-07, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.7495958957059526, + "rescued": 0, + "harmed": 10, + "missed_rescue": 1964, + "avoided_harm": 1161, + "selected_both_correct": 559, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": -3.3848826319626824e-07, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.7494553376906318, + "rescued": 0, + "harmed": 12, + "missed_rescue": 1964, + "avoided_harm": 1159, + "selected_both_correct": 699, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": -3.668622525765386e-07, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.7493850586829714, + "rescued": 0, + "harmed": 13, + "missed_rescue": 1964, + "avoided_harm": 1158, + "selected_both_correct": 840, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": -3.98026287484754e-07, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.749314779675311, + "rescued": 0, + "harmed": 14, + "missed_rescue": 1964, + "avoided_harm": 1157, + "selected_both_correct": 982, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": -4.302527258914779e-07, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.7491742216599901, + "rescued": 0, + "harmed": 16, + "missed_rescue": 1964, + "avoided_harm": 1155, + "selected_both_correct": 1122, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": -4.7191912244670675e-07, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.748963384637009, + "rescued": 0, + "harmed": 19, + "missed_rescue": 1964, + "avoided_harm": 1152, + "selected_both_correct": 1261, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": -5.115426802149159e-07, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.7486119895987069, + "rescued": 0, + "harmed": 24, + "missed_rescue": 1964, + "avoided_harm": 1147, + "selected_both_correct": 1398, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": -5.534793672268279e-07, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.7483308735680653, + "rescued": 0, + "harmed": 28, + "missed_rescue": 1964, + "avoided_harm": 1143, + "selected_both_correct": 1537, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": -5.941800509390305e-07, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.7481903155527444, + "rescued": 0, + "harmed": 30, + "missed_rescue": 1964, + "avoided_harm": 1141, + "selected_both_correct": 1677, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": -6.464504735959054e-07, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.7477686415067819, + "rescued": 0, + "harmed": 36, + "missed_rescue": 1964, + "avoided_harm": 1135, + "selected_both_correct": 1813, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": -6.999837864896108e-07, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.7476280834914611, + "rescued": 0, + "harmed": 38, + "missed_rescue": 1964, + "avoided_harm": 1133, + "selected_both_correct": 1954, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": -7.663418841730163e-07, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.7474875254761403, + "rescued": 0, + "harmed": 40, + "missed_rescue": 1964, + "avoided_harm": 1131, + "selected_both_correct": 2094, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": -8.471382670904859e-07, + "operator": "ge", + "actual_fraction": 0.15995502143509735, + "metric": 0.7472766884531591, + "rescued": 0, + "harmed": 43, + "missed_rescue": 1964, + "avoided_harm": 1128, + "selected_both_correct": 2233, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": -9.292756431023008e-07, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.7471361304378382, + "rescued": 0, + "harmed": 45, + "missed_rescue": 1964, + "avoided_harm": 1126, + "selected_both_correct": 2373, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": -1.0245233852401725e-06, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.7470658514301778, + "rescued": 0, + "harmed": 46, + "missed_rescue": 1964, + "avoided_harm": 1125, + "selected_both_correct": 2515, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": -1.1468601996966754e-06, + "operator": "ge", + "actual_fraction": 0.18996415770609318, + "metric": 0.7465036193688945, + "rescued": 0, + "harmed": 54, + "missed_rescue": 1964, + "avoided_harm": 1117, + "selected_both_correct": 2649, + "selected_both_wrong": 0 + }, + { + "budget": 0.2, + "threshold": -1.3018121762797819e-06, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.7462225033382529, + "rescued": 0, + "harmed": 58, + "missed_rescue": 1964, + "avoided_harm": 1113, + "selected_both_correct": 2787, + "selected_both_wrong": 0 + }, + { + "budget": 0.21, + "threshold": -1.4898060953782988e-06, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.7456602712769695, + "rescued": 0, + "harmed": 66, + "missed_rescue": 1964, + "avoided_harm": 1105, + "selected_both_correct": 2922, + "selected_both_wrong": 0 + }, + { + "budget": 0.22, + "threshold": -1.735574414851726e-06, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.7453088762386675, + "rescued": 0, + "harmed": 71, + "missed_rescue": 1964, + "avoided_harm": 1100, + "selected_both_correct": 3059, + "selected_both_wrong": 0 + }, + { + "budget": 0.23, + "threshold": -2.0881013824691763e-06, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.7446763651697238, + "rescued": 1, + "harmed": 81, + "missed_rescue": 1963, + "avoided_harm": 1090, + "selected_both_correct": 3190, + "selected_both_wrong": 0 + }, + { + "budget": 0.24, + "threshold": -2.5644872039265465e-06, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.7442546911237613, + "rescued": 1, + "harmed": 87, + "missed_rescue": 1963, + "avoided_harm": 1084, + "selected_both_correct": 3326, + "selected_both_wrong": 0 + }, + { + "budget": 0.25, + "threshold": -3.2399550491390983e-06, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.7437627380701385, + "rescued": 1, + "harmed": 94, + "missed_rescue": 1963, + "avoided_harm": 1077, + "selected_both_correct": 3462, + "selected_both_wrong": 0 + }, + { + "budget": 0.26, + "threshold": -4.2517635847616475e-06, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.7431302270011948, + "rescued": 1, + "harmed": 103, + "missed_rescue": 1963, + "avoided_harm": 1068, + "selected_both_correct": 3595, + "selected_both_wrong": 0 + }, + { + "budget": 0.27, + "threshold": -5.607922958006384e-06, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.7422868789092698, + "rescued": 1, + "harmed": 115, + "missed_rescue": 1963, + "avoided_harm": 1056, + "selected_both_correct": 3725, + "selected_both_wrong": 0 + }, + { + "budget": 0.28, + "threshold": -8.438555596512742e-06, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.7419354838709677, + "rescued": 1, + "harmed": 120, + "missed_rescue": 1963, + "avoided_harm": 1051, + "selected_both_correct": 3862, + "selected_both_wrong": 1 + }, + { + "budget": 0.29, + "threshold": -1.3145449884177651e-05, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.7411624147867032, + "rescued": 1, + "harmed": 131, + "missed_rescue": 1963, + "avoided_harm": 1040, + "selected_both_correct": 3993, + "selected_both_wrong": 1 + }, + { + "budget": 0.3, + "threshold": -2.076632881653495e-05, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.7404596247100991, + "rescued": 2, + "harmed": 142, + "missed_rescue": 1962, + "avoided_harm": 1029, + "selected_both_correct": 4123, + "selected_both_wrong": 1 + }, + { + "budget": 0.31, + "threshold": -4.0114435250870883e-05, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.7394757186028533, + "rescued": 2, + "harmed": 156, + "missed_rescue": 1962, + "avoided_harm": 1015, + "selected_both_correct": 4251, + "selected_both_wrong": 1 + }, + { + "budget": 0.32, + "threshold": -7.415730942739174e-05, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.7388432075339096, + "rescued": 2, + "harmed": 165, + "missed_rescue": 1962, + "avoided_harm": 1006, + "selected_both_correct": 4385, + "selected_both_wrong": 1 + }, + { + "budget": 0.33, + "threshold": -0.00015223574882838875, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.7377890224190035, + "rescued": 2, + "harmed": 180, + "missed_rescue": 1962, + "avoided_harm": 991, + "selected_both_correct": 4512, + "selected_both_wrong": 1 + }, + { + "budget": 0.34, + "threshold": -0.00032258269493468106, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.7361023262351536, + "rescued": 2, + "harmed": 204, + "missed_rescue": 1962, + "avoided_harm": 967, + "selected_both_correct": 4629, + "selected_both_wrong": 2 + }, + { + "budget": 0.35, + "threshold": -0.0006007538759149611, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.7348373040972661, + "rescued": 2, + "harmed": 222, + "missed_rescue": 1962, + "avoided_harm": 949, + "selected_both_correct": 4754, + "selected_both_wrong": 2 + }, + { + "budget": 0.36, + "threshold": -0.0011453087208792567, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.7337128399746996, + "rescued": 2, + "harmed": 238, + "missed_rescue": 1962, + "avoided_harm": 933, + "selected_both_correct": 4880, + "selected_both_wrong": 2 + }, + { + "budget": 0.37, + "threshold": -0.002608026610687375, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.732588375852133, + "rescued": 2, + "harmed": 254, + "missed_rescue": 1962, + "avoided_harm": 917, + "selected_both_correct": 5006, + "selected_both_wrong": 2 + }, + { + "budget": 0.38, + "threshold": -0.006219128146767616, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.7311827956989247, + "rescued": 2, + "harmed": 274, + "missed_rescue": 1962, + "avoided_harm": 897, + "selected_both_correct": 5128, + "selected_both_wrong": 3 + }, + { + "budget": 0.39, + "threshold": -0.01280172262340784, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.7297069365380561, + "rescued": 2, + "harmed": 295, + "missed_rescue": 1962, + "avoided_harm": 876, + "selected_both_correct": 5248, + "selected_both_wrong": 4 + }, + { + "budget": 0.4, + "threshold": -0.03794781118631363, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.7282310773771874, + "rescued": 3, + "harmed": 317, + "missed_rescue": 1961, + "avoided_harm": 854, + "selected_both_correct": 5365, + "selected_both_wrong": 6 + }, + { + "budget": 0.41, + "threshold": -0.08479184657335281, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.7266849392086584, + "rescued": 3, + "harmed": 339, + "missed_rescue": 1961, + "avoided_harm": 832, + "selected_both_correct": 5484, + "selected_both_wrong": 7 + }, + { + "budget": 0.42, + "threshold": -0.14941807091236115, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.7249279640171481, + "rescued": 7, + "harmed": 368, + "missed_rescue": 1957, + "avoided_harm": 803, + "selected_both_correct": 5590, + "selected_both_wrong": 11 + }, + { + "budget": 0.43, + "threshold": -0.2939925789833069, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.7233115468409586, + "rescued": 12, + "harmed": 396, + "missed_rescue": 1952, + "avoided_harm": 775, + "selected_both_correct": 5696, + "selected_both_wrong": 14 + }, + { + "budget": 0.44, + "threshold": -0.4386065602302551, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.7212734556188066, + "rescued": 15, + "harmed": 428, + "missed_rescue": 1949, + "avoided_harm": 743, + "selected_both_correct": 5798, + "selected_both_wrong": 19 + }, + { + "budget": 0.45, + "threshold": -0.5420411825180054, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.7190245273736735, + "rescued": 16, + "harmed": 461, + "missed_rescue": 1948, + "avoided_harm": 710, + "selected_both_correct": 5901, + "selected_both_wrong": 25 + }, + { + "budget": 0.46, + "threshold": -0.6230618953704834, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.7173378311898236, + "rescued": 21, + "harmed": 490, + "missed_rescue": 1943, + "avoided_harm": 681, + "selected_both_correct": 6008, + "selected_both_wrong": 26 + }, + { + "budget": 0.47, + "threshold": -0.6914289593696594, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.7155808559983133, + "rescued": 27, + "harmed": 521, + "missed_rescue": 1937, + "avoided_harm": 650, + "selected_both_correct": 6107, + "selected_both_wrong": 32 + }, + { + "budget": 0.48, + "threshold": -0.7387455701828003, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.7206409445498629, + "rescued": 105, + "harmed": 527, + "missed_rescue": 1859, + "avoided_harm": 644, + "selected_both_correct": 6125, + "selected_both_wrong": 72 + }, + { + "budget": 0.49, + "threshold": -0.7823225855827332, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.7280202403542062, + "rescued": 210, + "harmed": 527, + "missed_rescue": 1754, + "avoided_harm": 644, + "selected_both_correct": 6125, + "selected_both_wrong": 110 + }, + { + "budget": 0.5, + "threshold": -0.8206579089164734, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.7346264670742849, + "rescued": 305, + "harmed": 528, + "missed_rescue": 1659, + "avoided_harm": 643, + "selected_both_correct": 6125, + "selected_both_wrong": 156 + }, + { + "budget": 0.51, + "threshold": -0.8575630187988281, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.7414435308173448, + "rescued": 403, + "harmed": 529, + "missed_rescue": 1561, + "avoided_harm": 642, + "selected_both_correct": 6125, + "selected_both_wrong": 199 + }, + { + "budget": 0.52, + "threshold": -0.8823798894882202, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.7478389205144423, + "rescued": 495, + "harmed": 530, + "missed_rescue": 1469, + "avoided_harm": 641, + "selected_both_correct": 6125, + "selected_both_wrong": 249 + }, + { + "budget": 0.53, + "threshold": -0.9040287733078003, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.754093752196219, + "rescued": 585, + "harmed": 531, + "missed_rescue": 1379, + "avoided_harm": 640, + "selected_both_correct": 6125, + "selected_both_wrong": 300 + }, + { + "budget": 0.54, + "threshold": -0.9208687543869019, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.7602783048703352, + "rescued": 673, + "harmed": 531, + "missed_rescue": 1291, + "avoided_harm": 640, + "selected_both_correct": 6127, + "selected_both_wrong": 352 + }, + { + "budget": 0.55, + "threshold": -0.9344452619552612, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.7662520205214702, + "rescued": 759, + "harmed": 532, + "missed_rescue": 1205, + "avoided_harm": 639, + "selected_both_correct": 6129, + "selected_both_wrong": 405 + }, + { + "budget": 0.56, + "threshold": -0.9448322653770447, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.7718743411343032, + "rescued": 844, + "harmed": 537, + "missed_rescue": 1120, + "avoided_harm": 634, + "selected_both_correct": 6130, + "selected_both_wrong": 457 + }, + { + "budget": 0.57, + "threshold": -0.9531905055046082, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.7774263827394757, + "rescued": 924, + "harmed": 538, + "missed_rescue": 1040, + "avoided_harm": 633, + "selected_both_correct": 6133, + "selected_both_wrong": 515 + }, + { + "budget": 0.58, + "threshold": -0.9603281617164612, + "operator": "gt", + "actual_fraction": 0.5798720922060581, + "metric": 0.7818539602220816, + "rescued": 991, + "harmed": 542, + "missed_rescue": 973, + "avoided_harm": 629, + "selected_both_correct": 6135, + "selected_both_wrong": 583 + }, + { + "budget": 0.59, + "threshold": -0.9670801758766174, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.7864220957200084, + "rescued": 1061, + "harmed": 547, + "missed_rescue": 903, + "avoided_harm": 624, + "selected_both_correct": 6139, + "selected_both_wrong": 648 + }, + { + "budget": 0.6, + "threshold": -0.9724565148353577, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.7909902312179352, + "rescued": 1130, + "harmed": 551, + "missed_rescue": 834, + "avoided_harm": 620, + "selected_both_correct": 6146, + "selected_both_wrong": 710 + }, + { + "budget": 0.61, + "threshold": -0.9766654968261719, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.7954880877082016, + "rescued": 1200, + "harmed": 557, + "missed_rescue": 764, + "avoided_harm": 614, + "selected_both_correct": 6151, + "selected_both_wrong": 771 + }, + { + "budget": 0.62, + "threshold": -0.9801702499389648, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.7996345491601659, + "rescued": 1263, + "harmed": 561, + "missed_rescue": 701, + "avoided_harm": 610, + "selected_both_correct": 6159, + "selected_both_wrong": 838 + }, + { + "budget": 0.63, + "threshold": -0.9831460118293762, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.8028673835125448, + "rescued": 1317, + "harmed": 569, + "missed_rescue": 647, + "avoided_harm": 602, + "selected_both_correct": 6171, + "selected_both_wrong": 907 + }, + { + "budget": 0.64, + "threshold": -0.985751748085022, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.8064516129032258, + "rescued": 1380, + "harmed": 581, + "missed_rescue": 584, + "avoided_harm": 590, + "selected_both_correct": 6186, + "selected_both_wrong": 959 + }, + { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.809895284278586, + "rescued": 1434, + "harmed": 586, + "missed_rescue": 530, + "avoided_harm": 585, + "selected_both_correct": 6211, + "selected_both_wrong": 1017 + }, + { + "budget": 0.66, + "threshold": -0.9895955324172974, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.8123550495467003, + "rescued": 1479, + "harmed": 596, + "missed_rescue": 485, + "avoided_harm": 575, + "selected_both_correct": 6244, + "selected_both_wrong": 1072 + }, + { + "budget": 0.67, + "threshold": -0.9913432002067566, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.8149553728301356, + "rescued": 1525, + "harmed": 605, + "missed_rescue": 439, + "avoided_harm": 566, + "selected_both_correct": 6278, + "selected_both_wrong": 1125 + }, + { + "budget": 0.68, + "threshold": -0.9925730228424072, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.8169231850446271, + "rescued": 1569, + "harmed": 621, + "missed_rescue": 395, + "avoided_harm": 550, + "selected_both_correct": 6318, + "selected_both_wrong": 1167 + }, + { + "budget": 0.69, + "threshold": -0.9935283660888672, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.818609881228477, + "rescued": 1605, + "harmed": 633, + "missed_rescue": 359, + "avoided_harm": 538, + "selected_both_correct": 6376, + "selected_both_wrong": 1204 + }, + { + "budget": 0.7, + "threshold": -0.99436354637146, + "operator": "gt", + "actual_fraction": 0.6999086372900415, + "metric": 0.8196640663433832, + "rescued": 1636, + "harmed": 649, + "missed_rescue": 328, + "avoided_harm": 522, + "selected_both_correct": 6439, + "selected_both_wrong": 1235 + }, + { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.8201560193970061, + "rescued": 1660, + "harmed": 666, + "missed_rescue": 304, + "avoided_harm": 505, + "selected_both_correct": 6505, + "selected_both_wrong": 1271 + }, + { + "budget": 0.72, + "threshold": -0.9955723881721497, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.8200154613816852, + "rescued": 1686, + "harmed": 694, + "missed_rescue": 278, + "avoided_harm": 477, + "selected_both_correct": 6565, + "selected_both_wrong": 1299 + }, + { + "budget": 0.73, + "threshold": -0.9960037469863892, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.8199451823740249, + "rescued": 1703, + "harmed": 712, + "missed_rescue": 261, + "avoided_harm": 459, + "selected_both_correct": 6646, + "selected_both_wrong": 1326 + }, + { + "budget": 0.74, + "threshold": -0.9964693188667297, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.8200154613816852, + "rescued": 1722, + "harmed": 730, + "missed_rescue": 242, + "avoided_harm": 441, + "selected_both_correct": 6725, + "selected_both_wrong": 1352 + }, + { + "budget": 0.75, + "threshold": -0.9968041777610779, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.8198749033663645, + "rescued": 1742, + "harmed": 752, + "missed_rescue": 222, + "avoided_harm": 419, + "selected_both_correct": 6808, + "selected_both_wrong": 1369 + }, + { + "budget": 0.76, + "threshold": -0.9971042275428772, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.8200154613816852, + "rescued": 1763, + "harmed": 771, + "missed_rescue": 201, + "avoided_harm": 400, + "selected_both_correct": 6891, + "selected_both_wrong": 1389 + }, + { + "budget": 0.77, + "threshold": -0.9974007606506348, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.8195235083280624, + "rescued": 1784, + "harmed": 799, + "missed_rescue": 180, + "avoided_harm": 372, + "selected_both_correct": 6966, + "selected_both_wrong": 1407 + }, + { + "budget": 0.78, + "threshold": -0.9976577162742615, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.8191018342820999, + "rescued": 1800, + "harmed": 821, + "missed_rescue": 164, + "avoided_harm": 350, + "selected_both_correct": 7050, + "selected_both_wrong": 1427 + }, + { + "budget": 0.79, + "threshold": -0.9978766441345215, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.8192423922974208, + "rescued": 1818, + "harmed": 837, + "missed_rescue": 146, + "avoided_harm": 334, + "selected_both_correct": 7143, + "selected_both_wrong": 1442 + }, + { + "budget": 0.8, + "threshold": -0.9980920553207397, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.8190315552744395, + "rescued": 1834, + "harmed": 856, + "missed_rescue": 130, + "avoided_harm": 315, + "selected_both_correct": 7244, + "selected_both_wrong": 1449 + }, + { + "budget": 0.81, + "threshold": -0.9982843399047852, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.8190315552744395, + "rescued": 1849, + "harmed": 871, + "missed_rescue": 115, + "avoided_harm": 300, + "selected_both_correct": 7347, + "selected_both_wrong": 1458 + }, + { + "budget": 0.82, + "threshold": -0.9984900951385498, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.8187504392437979, + "rescued": 1859, + "harmed": 885, + "missed_rescue": 105, + "avoided_harm": 286, + "selected_both_correct": 7444, + "selected_both_wrong": 1479 + }, + { + "budget": 0.83, + "threshold": -0.9986395835876465, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.818609881228477, + "rescued": 1874, + "harmed": 902, + "missed_rescue": 90, + "avoided_harm": 269, + "selected_both_correct": 7541, + "selected_both_wrong": 1493 + }, + { + "budget": 0.84, + "threshold": -0.9987816214561462, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.8176962541288917, + "rescued": 1880, + "harmed": 921, + "missed_rescue": 84, + "avoided_harm": 250, + "selected_both_correct": 7643, + "selected_both_wrong": 1508 + }, + { + "budget": 0.85, + "threshold": -0.9989162087440491, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.8175556961135709, + "rescued": 1889, + "harmed": 932, + "missed_rescue": 75, + "avoided_harm": 239, + "selected_both_correct": 7754, + "selected_both_wrong": 1519 + }, + { + "budget": 0.86, + "threshold": -0.9990254044532776, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.8165717900063251, + "rescued": 1895, + "harmed": 952, + "missed_rescue": 69, + "avoided_harm": 219, + "selected_both_correct": 7864, + "selected_both_wrong": 1525 + }, + { + "budget": 0.87, + "threshold": -0.9991487264633179, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.8160798369527023, + "rescued": 1907, + "harmed": 971, + "missed_rescue": 57, + "avoided_harm": 200, + "selected_both_correct": 7969, + "selected_both_wrong": 1532 + }, + { + "budget": 0.88, + "threshold": -0.9992602467536926, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.8155878838990793, + "rescued": 1914, + "harmed": 985, + "missed_rescue": 50, + "avoided_harm": 186, + "selected_both_correct": 8082, + "selected_both_wrong": 1540 + }, + { + "budget": 0.89, + "threshold": -0.999350905418396, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.8148148148148148, + "rescued": 1919, + "harmed": 1001, + "missed_rescue": 45, + "avoided_harm": 170, + "selected_both_correct": 8192, + "selected_both_wrong": 1551 + }, + { + "budget": 0.9, + "threshold": -0.999435544013977, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.8144634197765127, + "rescued": 1927, + "harmed": 1014, + "missed_rescue": 37, + "avoided_harm": 157, + "selected_both_correct": 8308, + "selected_both_wrong": 1557 + }, + { + "budget": 0.91, + "threshold": -0.9995256066322327, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.8129172816079837, + "rescued": 1932, + "harmed": 1041, + "missed_rescue": 32, + "avoided_harm": 130, + "selected_both_correct": 8410, + "selected_both_wrong": 1565 + }, + { + "budget": 0.92, + "threshold": -0.9995898604393005, + "operator": "ge", + "actual_fraction": 0.9199522102747909, + "metric": 0.8123550495467003, + "rescued": 1937, + "harmed": 1054, + "missed_rescue": 27, + "avoided_harm": 117, + "selected_both_correct": 8530, + "selected_both_wrong": 1569 + }, + { + "budget": 0.93, + "threshold": -0.9996523857116699, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.8113711434394546, + "rescued": 1942, + "harmed": 1073, + "missed_rescue": 22, + "avoided_harm": 98, + "selected_both_correct": 8647, + "selected_both_wrong": 1570 + }, + { + "budget": 0.94, + "threshold": -0.9997095465660095, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.8102466793168881, + "rescued": 1945, + "harmed": 1092, + "missed_rescue": 19, + "avoided_harm": 79, + "selected_both_correct": 8764, + "selected_both_wrong": 1574 + }, + { + "budget": 0.95, + "threshold": -0.9997634887695312, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.8087708201560194, + "rescued": 1948, + "harmed": 1116, + "missed_rescue": 16, + "avoided_harm": 55, + "selected_both_correct": 8876, + "selected_both_wrong": 1577 + }, + { + "budget": 0.96, + "threshold": -0.9998065829277039, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.8079977510717549, + "rescued": 1951, + "harmed": 1130, + "missed_rescue": 13, + "avoided_harm": 41, + "selected_both_correct": 8998, + "selected_both_wrong": 1580 + }, + { + "budget": 0.97, + "threshold": -0.9998431205749512, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.8075057980181319, + "rescued": 1956, + "harmed": 1142, + "missed_rescue": 8, + "avoided_harm": 29, + "selected_both_correct": 9120, + "selected_both_wrong": 1584 + }, + { + "budget": 0.98, + "threshold": -0.9998764991760254, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.8070138449645091, + "rescued": 1961, + "harmed": 1154, + "missed_rescue": 3, + "avoided_harm": 17, + "selected_both_correct": 9242, + "selected_both_wrong": 1587 + }, + { + "budget": 0.99, + "threshold": -0.9999123811721802, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.806311054887905, + "rescued": 1962, + "harmed": 1165, + "missed_rescue": 2, + "avoided_harm": 6, + "selected_both_correct": 9370, + "selected_both_wrong": 1589 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8060299388572634, + "rescued": 1964, + "harmed": 1171, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9505, + "selected_both_wrong": 1589 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.6695119530385957, + "rescued": 0, + "harmed": 0, + "missed_rescue": 5423, + "avoided_harm": 2771, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -2.0005727208172175e-07, + "operator": "ge", + "actual_fraction": 0.002467175226157729, + "metric": 0.6686895612965431, + "rescued": 0, + "harmed": 29, + "missed_rescue": 5423, + "avoided_harm": 2742, + "selected_both_correct": 58, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -2.4156264544217265e-07, + "operator": "ge", + "actual_fraction": 0.004849275444516916, + "metric": 0.6680373195700876, + "rescued": 0, + "harmed": 52, + "missed_rescue": 5423, + "avoided_harm": 2719, + "selected_both_correct": 119, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -2.7729177531909954e-07, + "operator": "ge", + "actual_fraction": 0.007600034030003119, + "metric": 0.6675835861951621, + "rescued": 0, + "harmed": 68, + "missed_rescue": 5423, + "avoided_harm": 2703, + "selected_both_correct": 199, + "selected_both_wrong": 1 + }, + { + "budget": 0.04, + "threshold": -3.064858447032748e-07, + "operator": "ge", + "actual_fraction": 0.009811984232765221, + "metric": 0.6671014944843037, + "rescued": 0, + "harmed": 85, + "missed_rescue": 5423, + "avoided_harm": 2686, + "selected_both_correct": 260, + "selected_both_wrong": 1 + }, + { + "budget": 0.05, + "threshold": -3.3848826319626824e-07, + "operator": "ge", + "actual_fraction": 0.012534384482318578, + "metric": 0.6666194027734452, + "rescued": 0, + "harmed": 102, + "missed_rescue": 5423, + "avoided_harm": 2669, + "selected_both_correct": 338, + "selected_both_wrong": 2 + }, + { + "budget": 0.06, + "threshold": -3.668622525765386e-07, + "operator": "ge", + "actual_fraction": 0.01607917647392451, + "metric": 0.6662223860703854, + "rescued": 0, + "harmed": 116, + "missed_rescue": 5423, + "avoided_harm": 2655, + "selected_both_correct": 449, + "selected_both_wrong": 2 + }, + { + "budget": 0.07, + "threshold": -3.98026287484754e-07, + "operator": "ge", + "actual_fraction": 0.019709043473328986, + "metric": 0.6658253693673255, + "rescued": 0, + "harmed": 130, + "missed_rescue": 5423, + "avoided_harm": 2641, + "selected_both_correct": 562, + "selected_both_wrong": 3 + }, + { + "budget": 0.08, + "threshold": -4.302527258914779e-07, + "operator": "ge", + "actual_fraction": 0.02353741882426339, + "metric": 0.6655985026798628, + "rescued": 0, + "harmed": 138, + "missed_rescue": 5423, + "avoided_harm": 2633, + "selected_both_correct": 689, + "selected_both_wrong": 3 + }, + { + "budget": 0.09, + "threshold": -4.7191912244670675e-07, + "operator": "ge", + "actual_fraction": 0.028868785979638715, + "metric": 0.6651164109690043, + "rescued": 0, + "harmed": 155, + "missed_rescue": 5423, + "avoided_harm": 2616, + "selected_both_correct": 859, + "selected_both_wrong": 4 + }, + { + "budget": 0.1, + "threshold": -5.115426802149159e-07, + "operator": "ge", + "actual_fraction": 0.0345971698380739, + "metric": 0.6647761109378102, + "rescued": 0, + "harmed": 167, + "missed_rescue": 5423, + "avoided_harm": 2604, + "selected_both_correct": 1045, + "selected_both_wrong": 8 + }, + { + "budget": 0.11, + "threshold": -5.534793672268279e-07, + "operator": "ge", + "actual_fraction": 0.03981510364971783, + "metric": 0.664435810906616, + "rescued": 0, + "harmed": 179, + "missed_rescue": 5423, + "avoided_harm": 2592, + "selected_both_correct": 1215, + "selected_both_wrong": 10 + }, + { + "budget": 0.12, + "threshold": -5.941800509390305e-07, + "operator": "ge", + "actual_fraction": 0.045855429203414344, + "metric": 0.664067152539489, + "rescued": 0, + "harmed": 192, + "missed_rescue": 5423, + "avoided_harm": 2579, + "selected_both_correct": 1414, + "selected_both_wrong": 11 + }, + { + "budget": 0.13, + "threshold": -6.464504735959054e-07, + "operator": "ge", + "actual_fraction": 0.05317187987408899, + "metric": 0.6637552108442276, + "rescued": 1, + "harmed": 204, + "missed_rescue": 5422, + "avoided_harm": 2567, + "selected_both_correct": 1658, + "selected_both_wrong": 12 + }, + { + "budget": 0.14, + "threshold": -6.999837864896108e-07, + "operator": "ge", + "actual_fraction": 0.05960922213084536, + "metric": 0.6636134191645634, + "rescued": 1, + "harmed": 209, + "missed_rescue": 5422, + "avoided_harm": 2562, + "selected_both_correct": 1878, + "selected_both_wrong": 14 + }, + { + "budget": 0.15, + "threshold": -7.663418841730163e-07, + "operator": "ge", + "actual_fraction": 0.0675211978561098, + "metric": 0.6632164024615036, + "rescued": 2, + "harmed": 224, + "missed_rescue": 5421, + "avoided_harm": 2547, + "selected_both_correct": 2128, + "selected_both_wrong": 27 + }, + { + "budget": 0.16, + "threshold": -8.471382670904859e-07, + "operator": "ge", + "actual_fraction": 0.07551824858917279, + "metric": 0.6626208774069138, + "rescued": 2, + "harmed": 245, + "missed_rescue": 5421, + "avoided_harm": 2526, + "selected_both_correct": 2386, + "selected_both_wrong": 30 + }, + { + "budget": 0.17, + "threshold": -9.292756431023008e-07, + "operator": "ge", + "actual_fraction": 0.08212574086152624, + "metric": 0.6622522190397867, + "rescued": 3, + "harmed": 259, + "missed_rescue": 5420, + "avoided_harm": 2512, + "selected_both_correct": 2599, + "selected_both_wrong": 35 + }, + { + "budget": 0.18, + "threshold": -1.0245233852401725e-06, + "operator": "ge", + "actual_fraction": 0.08856308311828262, + "metric": 0.6619969940163911, + "rescued": 3, + "harmed": 268, + "missed_rescue": 5420, + "avoided_harm": 2503, + "selected_both_correct": 2812, + "selected_both_wrong": 40 + }, + { + "budget": 0.19, + "threshold": -1.1468601996966754e-06, + "operator": "ge", + "actual_fraction": 0.09480191702350906, + "metric": 0.6615149023055327, + "rescued": 3, + "harmed": 285, + "missed_rescue": 5420, + "avoided_harm": 2486, + "selected_both_correct": 3005, + "selected_both_wrong": 50 + }, + { + "budget": 0.2, + "threshold": -1.3018121762797819e-06, + "operator": "ge", + "actual_fraction": 0.10007656750701868, + "metric": 0.6612596772821371, + "rescued": 3, + "harmed": 294, + "missed_rescue": 5420, + "avoided_harm": 2477, + "selected_both_correct": 3174, + "selected_both_wrong": 58 + }, + { + "budget": 0.21, + "threshold": -1.4898060953782988e-06, + "operator": "ge", + "actual_fraction": 0.10554972634205825, + "metric": 0.6608343022431443, + "rescued": 3, + "harmed": 309, + "missed_rescue": 5420, + "avoided_harm": 2462, + "selected_both_correct": 3340, + "selected_both_wrong": 70 + }, + { + "budget": 0.22, + "threshold": -1.735574414851726e-06, + "operator": "ge", + "actual_fraction": 0.11005870175538099, + "metric": 0.6605507188838159, + "rescued": 3, + "harmed": 319, + "missed_rescue": 5420, + "avoided_harm": 2452, + "selected_both_correct": 3485, + "selected_both_wrong": 74 + }, + { + "budget": 0.23, + "threshold": -2.0881013824691763e-06, + "operator": "ge", + "actual_fraction": 0.11527663556702493, + "metric": 0.6600969855088903, + "rescued": 5, + "harmed": 337, + "missed_rescue": 5418, + "avoided_harm": 2434, + "selected_both_correct": 3630, + "selected_both_wrong": 93 + }, + { + "budget": 0.24, + "threshold": -2.5644872039265465e-06, + "operator": "ge", + "actual_fraction": 0.12029606102713893, + "metric": 0.6598417604854947, + "rescued": 8, + "harmed": 349, + "missed_rescue": 5415, + "avoided_harm": 2422, + "selected_both_correct": 3769, + "selected_both_wrong": 116 + }, + { + "budget": 0.25, + "threshold": -3.2399550491390983e-06, + "operator": "ge", + "actual_fraction": 0.12477667810452883, + "metric": 0.6594731021183677, + "rescued": 8, + "harmed": 362, + "missed_rescue": 5415, + "avoided_harm": 2409, + "selected_both_correct": 3894, + "selected_both_wrong": 136 + }, + { + "budget": 0.26, + "threshold": -4.2517635847616475e-06, + "operator": "ge", + "actual_fraction": 0.12914386183818732, + "metric": 0.6590193687434421, + "rescued": 8, + "harmed": 378, + "missed_rescue": 5415, + "avoided_harm": 2393, + "selected_both_correct": 4012, + "selected_both_wrong": 156 + }, + { + "budget": 0.27, + "threshold": -5.607922958006384e-06, + "operator": "ge", + "actual_fraction": 0.1322916371267334, + "metric": 0.658679068712248, + "rescued": 10, + "harmed": 392, + "missed_rescue": 5413, + "avoided_harm": 2379, + "selected_both_correct": 4092, + "selected_both_wrong": 171 + }, + { + "budget": 0.28, + "threshold": -8.438555596512742e-06, + "operator": "ge", + "actual_fraction": 0.13626180415733205, + "metric": 0.6582820520091881, + "rescued": 12, + "harmed": 408, + "missed_rescue": 5411, + "avoided_harm": 2363, + "selected_both_correct": 4192, + "selected_both_wrong": 193 + }, + { + "budget": 0.29, + "threshold": -1.3145449884177651e-05, + "operator": "ge", + "actual_fraction": 0.14023197118793068, + "metric": 0.657743243626464, + "rescued": 13, + "harmed": 428, + "missed_rescue": 5410, + "avoided_harm": 2343, + "selected_both_correct": 4285, + "selected_both_wrong": 219 + }, + { + "budget": 0.3, + "threshold": -2.076632881653495e-05, + "operator": "ge", + "actual_fraction": 0.14326631313274538, + "metric": 0.657374585259337, + "rescued": 16, + "harmed": 444, + "missed_rescue": 5407, + "avoided_harm": 2327, + "selected_both_correct": 4359, + "selected_both_wrong": 233 + }, + { + "budget": 0.31, + "threshold": -4.0114435250870883e-05, + "operator": "ge", + "actual_fraction": 0.14766185520233674, + "metric": 0.6569775685562771, + "rescued": 18, + "harmed": 460, + "missed_rescue": 5405, + "avoided_harm": 2311, + "selected_both_correct": 4464, + "selected_both_wrong": 265 + }, + { + "budget": 0.32, + "threshold": -7.415730942739174e-05, + "operator": "ge", + "actual_fraction": 0.15066783881121856, + "metric": 0.6567507018688143, + "rescued": 20, + "harmed": 470, + "missed_rescue": 5403, + "avoided_harm": 2301, + "selected_both_correct": 4545, + "selected_both_wrong": 278 + }, + { + "budget": 0.33, + "threshold": -0.00015223574882838875, + "operator": "ge", + "actual_fraction": 0.15452457249808582, + "metric": 0.6565238351813516, + "rescued": 27, + "harmed": 485, + "missed_rescue": 5396, + "avoided_harm": 2286, + "selected_both_correct": 4632, + "selected_both_wrong": 305 + }, + { + "budget": 0.34, + "threshold": -0.00032258269493468106, + "operator": "ge", + "actual_fraction": 0.15806936448969175, + "metric": 0.6564104018376201, + "rescued": 39, + "harmed": 501, + "missed_rescue": 5384, + "avoided_harm": 2270, + "selected_both_correct": 4705, + "selected_both_wrong": 329 + }, + { + "budget": 0.35, + "threshold": -0.0006007538759149611, + "operator": "ge", + "actual_fraction": 0.16201117318435754, + "metric": 0.6562686101579559, + "rescued": 46, + "harmed": 513, + "missed_rescue": 5377, + "avoided_harm": 2258, + "selected_both_correct": 4777, + "selected_both_wrong": 377 + }, + { + "budget": 0.36, + "threshold": -0.0011453087208792567, + "operator": "ge", + "actual_fraction": 0.1657544735274934, + "metric": 0.6558999517908289, + "rescued": 48, + "harmed": 528, + "missed_rescue": 5375, + "avoided_harm": 2243, + "selected_both_correct": 4841, + "selected_both_wrong": 428 + }, + { + "budget": 0.37, + "threshold": -0.002608026610687375, + "operator": "ge", + "actual_fraction": 0.17012165726115192, + "metric": 0.6558715934548961, + "rescued": 61, + "harmed": 542, + "missed_rescue": 5362, + "avoided_harm": 2229, + "selected_both_correct": 4937, + "selected_both_wrong": 459 + }, + { + "budget": 0.38, + "threshold": -0.006219128146767616, + "operator": "ge", + "actual_fraction": 0.17536794940872868, + "metric": 0.6555596517596347, + "rescued": 79, + "harmed": 571, + "missed_rescue": 5344, + "avoided_harm": 2200, + "selected_both_correct": 5018, + "selected_both_wrong": 516 + }, + { + "budget": 0.39, + "threshold": -0.01280172262340784, + "operator": "ge", + "actual_fraction": 0.17965005813458867, + "metric": 0.6553044267362391, + "rescued": 92, + "harmed": 593, + "missed_rescue": 5331, + "avoided_harm": 2178, + "selected_both_correct": 5091, + "selected_both_wrong": 559 + }, + { + "budget": 0.4, + "threshold": -0.03794781118631363, + "operator": "ge", + "actual_fraction": 0.18401724186824717, + "metric": 0.6552193517284406, + "rescued": 107, + "harmed": 611, + "missed_rescue": 5316, + "avoided_harm": 2160, + "selected_both_correct": 5145, + "selected_both_wrong": 626 + }, + { + "budget": 0.41, + "threshold": -0.08479184657335281, + "operator": "ge", + "actual_fraction": 0.18858293395343562, + "metric": 0.6552760684003063, + "rescued": 127, + "harmed": 629, + "missed_rescue": 5296, + "avoided_harm": 2142, + "selected_both_correct": 5207, + "selected_both_wrong": 687 + }, + { + "budget": 0.42, + "threshold": -0.14941807091236115, + "operator": "ge", + "actual_fraction": 0.19238295096843716, + "metric": 0.6554745767518362, + "rescued": 144, + "harmed": 639, + "missed_rescue": 5279, + "avoided_harm": 2132, + "selected_both_correct": 5277, + "selected_both_wrong": 724 + }, + { + "budget": 0.43, + "threshold": -0.2939925789833069, + "operator": "ge", + "actual_fraction": 0.19689192638175992, + "metric": 0.655502935087769, + "rescued": 168, + "harmed": 662, + "missed_rescue": 5255, + "avoided_harm": 2109, + "selected_both_correct": 5344, + "selected_both_wrong": 769 + }, + { + "budget": 0.44, + "threshold": -0.4386065602302551, + "operator": "ge", + "actual_fraction": 0.20182627683407536, + "metric": 0.6555880100955676, + "rescued": 199, + "harmed": 690, + "missed_rescue": 5224, + "avoided_harm": 2081, + "selected_both_correct": 5429, + "selected_both_wrong": 799 + }, + { + "budget": 0.45, + "threshold": -0.5420411825180054, + "operator": "ge", + "actual_fraction": 0.20718600232538353, + "metric": 0.6559283101267618, + "rescued": 240, + "harmed": 719, + "missed_rescue": 5183, + "avoided_harm": 2052, + "selected_both_correct": 5506, + "selected_both_wrong": 841 + }, + { + "budget": 0.46, + "threshold": -0.6230618953704834, + "operator": "ge", + "actual_fraction": 0.21189348609023623, + "metric": 0.656070101806426, + "rescued": 280, + "harmed": 754, + "missed_rescue": 5143, + "avoided_harm": 2017, + "selected_both_correct": 5554, + "selected_both_wrong": 884 + }, + { + "budget": 0.47, + "threshold": -0.6914289593696594, + "operator": "ge", + "actual_fraction": 0.217139778237813, + "metric": 0.6567790602047472, + "rescued": 329, + "harmed": 778, + "missed_rescue": 5094, + "avoided_harm": 1993, + "selected_both_correct": 5614, + "selected_both_wrong": 936 + }, + { + "budget": 0.48, + "threshold": -0.7387455701828003, + "operator": "ge", + "actual_fraction": 0.22198905368232993, + "metric": 0.6584238436888523, + "rescued": 399, + "harmed": 790, + "missed_rescue": 5024, + "avoided_harm": 1981, + "selected_both_correct": 5648, + "selected_both_wrong": 991 + }, + { + "budget": 0.49, + "threshold": -0.7823225855827332, + "operator": "ge", + "actual_fraction": 0.22763236253296656, + "metric": 0.6604372855400845, + "rescued": 483, + "harmed": 803, + "missed_rescue": 4940, + "avoided_harm": 1968, + "selected_both_correct": 5689, + "selected_both_wrong": 1052 + }, + { + "budget": 0.5, + "threshold": -0.8206579089164734, + "operator": "ge", + "actual_fraction": 0.2345234381646485, + "metric": 0.662762669086578, + "rescued": 579, + "harmed": 817, + "missed_rescue": 4844, + "avoided_harm": 1954, + "selected_both_correct": 5735, + "selected_both_wrong": 1139 + }, + { + "budget": 0.51, + "threshold": -0.8575630187988281, + "operator": "ge", + "actual_fraction": 0.24382497235062248, + "metric": 0.6654567110001985, + "rescued": 702, + "harmed": 845, + "missed_rescue": 4721, + "avoided_harm": 1926, + "selected_both_correct": 5794, + "selected_both_wrong": 1257 + }, + { + "budget": 0.52, + "threshold": -0.8823798894882202, + "operator": "ge", + "actual_fraction": 0.252644414825738, + "metric": 0.6678955278904234, + "rescued": 813, + "harmed": 870, + "missed_rescue": 4610, + "avoided_harm": 1901, + "selected_both_correct": 5842, + "selected_both_wrong": 1384 + }, + { + "budget": 0.53, + "threshold": -0.9040287733078003, + "operator": "ge", + "actual_fraction": 0.26401610753480986, + "metric": 0.6703627031165811, + "rescued": 937, + "harmed": 907, + "missed_rescue": 4486, + "avoided_harm": 1864, + "selected_both_correct": 5946, + "selected_both_wrong": 1520 + }, + { + "budget": 0.54, + "threshold": -0.9208687543869019, + "operator": "ge", + "actual_fraction": 0.2760116836344043, + "metric": 0.6731134617020673, + "rescued": 1067, + "harmed": 940, + "missed_rescue": 4356, + "avoided_harm": 1831, + "selected_both_correct": 6068, + "selected_both_wrong": 1658 + }, + { + "budget": 0.55, + "threshold": -0.9344452619552612, + "operator": "ge", + "actual_fraction": 0.2893401015228426, + "metric": 0.6758358619516207, + "rescued": 1197, + "harmed": 974, + "missed_rescue": 4226, + "avoided_harm": 1797, + "selected_both_correct": 6218, + "selected_both_wrong": 1814 + }, + { + "budget": 0.56, + "threshold": -0.9448322653770447, + "operator": "ge", + "actual_fraction": 0.3036043444970649, + "metric": 0.679295578935428, + "rescued": 1352, + "harmed": 1007, + "missed_rescue": 4071, + "avoided_harm": 1764, + "selected_both_correct": 6385, + "selected_both_wrong": 1962 + }, + { + "budget": 0.57, + "threshold": -0.9531905055046082, + "operator": "ge", + "actual_fraction": 0.3175850041119587, + "metric": 0.6822448458724442, + "rescued": 1489, + "harmed": 1040, + "missed_rescue": 3934, + "avoided_harm": 1731, + "selected_both_correct": 6573, + "selected_both_wrong": 2097 + }, + { + "budget": 0.58, + "threshold": -0.9603281617164612, + "operator": "gt", + "actual_fraction": 0.33462836400760004, + "metric": 0.6853926211609903, + "rescued": 1645, + "harmed": 1085, + "missed_rescue": 3778, + "avoided_harm": 1686, + "selected_both_correct": 6816, + "selected_both_wrong": 2254 + }, + { + "budget": 0.59, + "threshold": -0.9670801758766174, + "operator": "ge", + "actual_fraction": 0.35637920766809406, + "metric": 0.6889090548166633, + "rescued": 1818, + "harmed": 1134, + "missed_rescue": 3605, + "avoided_harm": 1637, + "selected_both_correct": 7175, + "selected_both_wrong": 2440 + }, + { + "budget": 0.6, + "threshold": -0.9724565148353577, + "operator": "ge", + "actual_fraction": 0.37968975980489467, + "metric": 0.6922553384567394, + "rescued": 1978, + "harmed": 1176, + "missed_rescue": 3445, + "avoided_harm": 1595, + "selected_both_correct": 7569, + "selected_both_wrong": 2666 + }, + { + "budget": 0.61, + "threshold": -0.9766654968261719, + "operator": "ge", + "actual_fraction": 0.4046450954258004, + "metric": 0.6949493803703599, + "rescued": 2126, + "harmed": 1229, + "missed_rescue": 3297, + "avoided_harm": 1542, + "selected_both_correct": 8013, + "selected_both_wrong": 2901 + }, + { + "budget": 0.62, + "threshold": -0.9801702499389648, + "operator": "ge", + "actual_fraction": 0.43141536454640844, + "metric": 0.6983807390182344, + "rescued": 2296, + "harmed": 1278, + "missed_rescue": 3127, + "avoided_harm": 1493, + "selected_both_correct": 8524, + "selected_both_wrong": 3115 + }, + { + "budget": 0.63, + "threshold": -0.9831460118293762, + "operator": "ge", + "actual_fraction": 0.4610214672603012, + "metric": 0.7015568726427134, + "rescued": 2479, + "harmed": 1349, + "missed_rescue": 2944, + "avoided_harm": 1422, + "selected_both_correct": 9071, + "selected_both_wrong": 3358 + }, + { + "budget": 0.64, + "threshold": -0.985751748085022, + "operator": "ge", + "actual_fraction": 0.4916484700677764, + "metric": 0.7053285313217821, + "rescued": 2674, + "harmed": 1411, + "missed_rescue": 2749, + "avoided_harm": 1360, + "selected_both_correct": 9690, + "selected_both_wrong": 3562 + }, + { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.5209426310864078, + "metric": 0.7086180982899923, + "rescued": 2849, + "harmed": 1470, + "missed_rescue": 2574, + "avoided_harm": 1301, + "selected_both_correct": 10321, + "selected_both_wrong": 3730 + }, + { + "budget": 0.66, + "threshold": -0.9895955324172974, + "operator": "ge", + "actual_fraction": 0.5529308340186598, + "metric": 0.71148229021921, + "rescued": 3014, + "harmed": 1534, + "missed_rescue": 2409, + "avoided_harm": 1237, + "selected_both_correct": 11019, + "selected_both_wrong": 3931 + }, + { + "budget": 0.67, + "threshold": -0.9913432002067566, + "operator": "ge", + "actual_fraction": 0.5888041289737118, + "metric": 0.7155091739216742, + "rescued": 3227, + "harmed": 1605, + "missed_rescue": 2196, + "avoided_harm": 1166, + "selected_both_correct": 11762, + "selected_both_wrong": 4169 + }, + { + "budget": 0.68, + "threshold": -0.9925730228424072, + "operator": "ge", + "actual_fraction": 0.6175027649377535, + "metric": 0.7182882908430933, + "rescued": 3404, + "harmed": 1684, + "missed_rescue": 2019, + "avoided_harm": 1087, + "selected_both_correct": 12326, + "selected_both_wrong": 4361 + }, + { + "budget": 0.69, + "threshold": -0.9935283660888672, + "operator": "ge", + "actual_fraction": 0.6421461588633979, + "metric": 0.7211241244363781, + "rescued": 3553, + "harmed": 1733, + "missed_rescue": 1870, + "avoided_harm": 1038, + "selected_both_correct": 12849, + "selected_both_wrong": 4509 + }, + { + "budget": 0.7, + "threshold": -0.99436354637146, + "operator": "gt", + "actual_fraction": 0.665910444375124, + "metric": 0.7233077163032073, + "rescued": 3694, + "harmed": 1797, + "missed_rescue": 1729, + "avoided_harm": 974, + "selected_both_correct": 13314, + "selected_both_wrong": 4677 + }, + { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.6838612710206166, + "metric": 0.7243286163967898, + "rescued": 3794, + "harmed": 1861, + "missed_rescue": 1629, + "avoided_harm": 910, + "selected_both_correct": 13678, + "selected_both_wrong": 4782 + }, + { + "budget": 0.72, + "threshold": -0.9955723881721497, + "operator": "ge", + "actual_fraction": 0.7029464311034228, + "metric": 0.7260017582168279, + "rescued": 3908, + "harmed": 1916, + "missed_rescue": 1515, + "avoided_harm": 855, + "selected_both_correct": 14058, + "selected_both_wrong": 4906 + }, + { + "budget": 0.73, + "threshold": -0.9960037469863892, + "operator": "ge", + "actual_fraction": 0.7180614241556306, + "metric": 0.7278450500524629, + "rescued": 4015, + "harmed": 1958, + "missed_rescue": 1408, + "avoided_harm": 813, + "selected_both_correct": 14354, + "selected_both_wrong": 4994 + }, + { + "budget": 0.74, + "threshold": -0.9964693188667297, + "operator": "ge", + "actual_fraction": 0.7352465757309361, + "metric": 0.7297450585599637, + "rescued": 4134, + "harmed": 2010, + "missed_rescue": 1289, + "avoided_harm": 761, + "selected_both_correct": 14689, + "selected_both_wrong": 5094 + }, + { + "budget": 0.75, + "threshold": -0.9968041777610779, + "operator": "ge", + "actual_fraction": 0.7478093185491875, + "metric": 0.7311629753566061, + "rescued": 4208, + "harmed": 2034, + "missed_rescue": 1215, + "avoided_harm": 737, + "selected_both_correct": 14961, + "selected_both_wrong": 5167 + }, + { + "budget": 0.76, + "threshold": -0.9971042275428772, + "operator": "ge", + "actual_fraction": 0.759804894648782, + "metric": 0.7328928338485098, + "rescued": 4299, + "harmed": 2064, + "missed_rescue": 1124, + "avoided_harm": 707, + "selected_both_correct": 15186, + "selected_both_wrong": 5244 + }, + { + "budget": 0.77, + "threshold": -0.9974007606506348, + "operator": "ge", + "actual_fraction": 0.7729064458497575, + "metric": 0.7342540339732865, + "rescued": 4385, + "harmed": 2102, + "missed_rescue": 1038, + "avoided_harm": 669, + "selected_both_correct": 15447, + "selected_both_wrong": 5321 + }, + { + "budget": 0.78, + "threshold": -0.9976577162742615, + "operator": "ge", + "actual_fraction": 0.7855259053398747, + "metric": 0.735274934066869, + "rescued": 4462, + "harmed": 2143, + "missed_rescue": 961, + "avoided_harm": 628, + "selected_both_correct": 15703, + "selected_both_wrong": 5392 + }, + { + "budget": 0.79, + "threshold": -0.9978766441345215, + "operator": "ge", + "actual_fraction": 0.7964722230099538, + "metric": 0.7362958341604514, + "rescued": 4527, + "harmed": 2172, + "missed_rescue": 896, + "avoided_harm": 599, + "selected_both_correct": 15920, + "selected_both_wrong": 5467 + }, + { + "budget": 0.8, + "threshold": -0.9980920553207397, + "operator": "ge", + "actual_fraction": 0.8082125740861527, + "metric": 0.7377704676289595, + "rescued": 4609, + "harmed": 2202, + "missed_rescue": 814, + "avoided_harm": 569, + "selected_both_correct": 16159, + "selected_both_wrong": 5530 + }, + { + "budget": 0.81, + "threshold": -0.9982843399047852, + "operator": "ge", + "actual_fraction": 0.8197544168108215, + "metric": 0.738422709355415, + "rescued": 4678, + "harmed": 2248, + "missed_rescue": 745, + "avoided_harm": 523, + "selected_both_correct": 16389, + "selected_both_wrong": 5592 + }, + { + "budget": 0.82, + "threshold": -0.9984900951385498, + "operator": "ge", + "actual_fraction": 0.8333380597226555, + "metric": 0.7389048010662734, + "rescued": 4746, + "harmed": 2299, + "missed_rescue": 677, + "avoided_harm": 472, + "selected_both_correct": 16674, + "selected_both_wrong": 5667 + }, + { + "budget": 0.83, + "threshold": -0.9986395835876465, + "operator": "ge", + "actual_fraction": 0.8426112355726966, + "metric": 0.7398689844879902, + "rescued": 4807, + "harmed": 2326, + "missed_rescue": 616, + "avoided_harm": 445, + "selected_both_correct": 16857, + "selected_both_wrong": 5723 + }, + { + "budget": 0.84, + "threshold": -0.9987816214561462, + "operator": "ge", + "actual_fraction": 0.8529903865241187, + "metric": 0.7403227178629158, + "rescued": 4859, + "harmed": 2362, + "missed_rescue": 564, + "avoided_harm": 409, + "selected_both_correct": 17089, + "selected_both_wrong": 5769 + }, + { + "budget": 0.85, + "threshold": -0.9989162087440491, + "operator": "ge", + "actual_fraction": 0.8635396874911381, + "metric": 0.7406913762300428, + "rescued": 4914, + "harmed": 2404, + "missed_rescue": 509, + "avoided_harm": 367, + "selected_both_correct": 17300, + "selected_both_wrong": 5833 + }, + { + "budget": 0.86, + "threshold": -0.9990254044532776, + "operator": "ge", + "actual_fraction": 0.8710262881774098, + "metric": 0.7413152596205654, + "rescued": 4962, + "harmed": 2430, + "missed_rescue": 461, + "avoided_harm": 341, + "selected_both_correct": 17455, + "selected_both_wrong": 5868 + }, + { + "budget": 0.87, + "threshold": -0.9991487264633179, + "operator": "ge", + "actual_fraction": 0.8809517057539064, + "metric": 0.7422227263704165, + "rescued": 5018, + "harmed": 2454, + "missed_rescue": 405, + "avoided_harm": 317, + "selected_both_correct": 17666, + "selected_both_wrong": 5927 + }, + { + "budget": 0.88, + "threshold": -0.9992602467536926, + "operator": "ge", + "actual_fraction": 0.8913024983693957, + "metric": 0.7430167597765364, + "rescued": 5074, + "harmed": 2482, + "missed_rescue": 349, + "avoided_harm": 289, + "selected_both_correct": 17892, + "selected_both_wrong": 5982 + }, + { + "budget": 0.89, + "threshold": -0.999350905418396, + "operator": "ge", + "actual_fraction": 0.900915974250631, + "metric": 0.743442134815529, + "rescued": 5120, + "harmed": 2513, + "missed_rescue": 303, + "avoided_harm": 258, + "selected_both_correct": 18112, + "selected_both_wrong": 6024 + }, + { + "budget": 0.9, + "threshold": -0.999435544013977, + "operator": "ge", + "actual_fraction": 0.910954825170859, + "metric": 0.7443212432294473, + "rescued": 5172, + "harmed": 2534, + "missed_rescue": 251, + "avoided_harm": 237, + "selected_both_correct": 18362, + "selected_both_wrong": 6055 + }, + { + "budget": 0.91, + "threshold": -0.9995256066322327, + "operator": "ge", + "actual_fraction": 0.921901142840938, + "metric": 0.7448033349403057, + "rescued": 5223, + "harmed": 2568, + "missed_rescue": 200, + "avoided_harm": 203, + "selected_both_correct": 18626, + "selected_both_wrong": 6092 + }, + { + "budget": 0.92, + "threshold": -0.9995898604393005, + "operator": "ge", + "actual_fraction": 0.9303235686129938, + "metric": 0.7450018432918356, + "rescued": 5253, + "harmed": 2591, + "missed_rescue": 170, + "avoided_harm": 180, + "selected_both_correct": 18846, + "selected_both_wrong": 6116 + }, + { + "budget": 0.93, + "threshold": -0.9996523857116699, + "operator": "ge", + "actual_fraction": 0.9393698777755721, + "metric": 0.7448884099481042, + "rescued": 5284, + "harmed": 2626, + "missed_rescue": 139, + "avoided_harm": 145, + "selected_both_correct": 19077, + "selected_both_wrong": 6138 + }, + { + "budget": 0.94, + "threshold": -0.9997095465660095, + "operator": "ge", + "actual_fraction": 0.9490117119927403, + "metric": 0.7448884099481042, + "rescued": 5314, + "harmed": 2656, + "missed_rescue": 109, + "avoided_harm": 115, + "selected_both_correct": 19329, + "selected_both_wrong": 6166 + }, + { + "budget": 0.95, + "threshold": -0.9997634887695312, + "operator": "ge", + "actual_fraction": 0.9587953378895726, + "metric": 0.7449734849559028, + "rescued": 5347, + "harmed": 2686, + "missed_rescue": 76, + "avoided_harm": 85, + "selected_both_correct": 19588, + "selected_both_wrong": 6189 + }, + { + "budget": 0.96, + "threshold": -0.9998065829277039, + "operator": "ge", + "actual_fraction": 0.9673595553412926, + "metric": 0.7452570683152313, + "rescued": 5375, + "harmed": 2704, + "missed_rescue": 48, + "avoided_harm": 67, + "selected_both_correct": 19828, + "selected_both_wrong": 6205 + }, + { + "budget": 0.97, + "threshold": -0.9998431205749512, + "operator": "ge", + "actual_fraction": 0.9746192893401016, + "metric": 0.745313784987097, + "rescued": 5398, + "harmed": 2725, + "missed_rescue": 25, + "avoided_harm": 46, + "selected_both_correct": 20033, + "selected_both_wrong": 6212 + }, + { + "budget": 0.98, + "threshold": -0.9998764991760254, + "operator": "ge", + "actual_fraction": 0.9830984317840229, + "metric": 0.7450585599637013, + "rescued": 5411, + "harmed": 2747, + "missed_rescue": 12, + "avoided_harm": 24, + "selected_both_correct": 20290, + "selected_both_wrong": 6219 + }, + { + "budget": 0.99, + "threshold": -0.9999123811721802, + "operator": "ge", + "actual_fraction": 0.9908118991577575, + "metric": 0.7448884099481042, + "rescued": 5419, + "harmed": 2761, + "missed_rescue": 4, + "avoided_harm": 10, + "selected_both_correct": 20535, + "selected_both_wrong": 6224 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.7447182599325072, + "rescued": 5423, + "harmed": 2771, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 20838, + "selected_both_wrong": 6231 + } + ] + }, + "learned": { + "selected_budget_index": 56, + "validation_area_0_25": 0.777812917281608, + "validation_point": { + "budget": 0.56, + "threshold": -5.435944080352783, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.8081383090870757, + "rescued": 1513, + "harmed": 690, + "missed_rescue": 451, + "avoided_harm": 481, + "selected_both_correct": 4599, + "selected_both_wrong": 1166 + }, + "test_point": { + "budget": 0.56, + "threshold": -5.435944080352783, + "operator": "ge", + "actual_fraction": 0.5355755324277571, + "metric": 0.7528287440093016, + "rescued": 4361, + "harmed": 1423, + "missed_rescue": 1062, + "avoided_harm": 1348, + "selected_both_correct": 8197, + "selected_both_wrong": 4905 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7502986857825568, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1964, + "avoided_harm": 1171, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 5.873929023742676, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.7559210063953897, + "rescued": 80, + "harmed": 0, + "missed_rescue": 1884, + "avoided_harm": 1171, + "selected_both_correct": 1, + "selected_both_wrong": 61 + }, + { + "budget": 0.02, + "threshold": 3.9572336673736572, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.7606999789162977, + "rescued": 149, + "harmed": 1, + "missed_rescue": 1815, + "avoided_harm": 1170, + "selected_both_correct": 3, + "selected_both_wrong": 131 + }, + { + "budget": 0.03, + "threshold": 2.499727725982666, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.765197835406564, + "rescued": 217, + "harmed": 5, + "missed_rescue": 1747, + "avoided_harm": 1166, + "selected_both_correct": 16, + "selected_both_wrong": 188 + }, + { + "budget": 0.04, + "threshold": 1.2223308086395264, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.7671656476210555, + "rescued": 258, + "harmed": 18, + "missed_rescue": 1706, + "avoided_harm": 1153, + "selected_both_correct": 51, + "selected_both_wrong": 242 + }, + { + "budget": 0.05, + "threshold": 0.36731216311454773, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.767587321667018, + "rescued": 283, + "harmed": 37, + "missed_rescue": 1681, + "avoided_harm": 1134, + "selected_both_correct": 113, + "selected_both_wrong": 278 + }, + { + "budget": 0.06, + "threshold": -0.14144867658615112, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.7663925785367911, + "rescued": 293, + "harmed": 64, + "missed_rescue": 1671, + "avoided_harm": 1107, + "selected_both_correct": 199, + "selected_both_wrong": 297 + }, + { + "budget": 0.07, + "threshold": -0.42117512226104736, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.7665331365521119, + "rescued": 320, + "harmed": 89, + "missed_rescue": 1644, + "avoided_harm": 1082, + "selected_both_correct": 259, + "selected_both_wrong": 328 + }, + { + "budget": 0.08, + "threshold": -0.6775990128517151, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.7686415067819242, + "rescued": 368, + "harmed": 107, + "missed_rescue": 1596, + "avoided_harm": 1064, + "selected_both_correct": 304, + "selected_both_wrong": 359 + }, + { + "budget": 0.09, + "threshold": -0.9111056327819824, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.7707498770117366, + "rescued": 412, + "harmed": 121, + "missed_rescue": 1552, + "avoided_harm": 1050, + "selected_both_correct": 343, + "selected_both_wrong": 404 + }, + { + "budget": 0.1, + "threshold": -1.1332147121429443, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.772858247241549, + "rescued": 454, + "harmed": 133, + "missed_rescue": 1510, + "avoided_harm": 1038, + "selected_both_correct": 393, + "selected_both_wrong": 442 + }, + { + "budget": 0.11, + "threshold": -1.3457791805267334, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.774404385410078, + "rescued": 491, + "harmed": 148, + "missed_rescue": 1473, + "avoided_harm": 1023, + "selected_both_correct": 446, + "selected_both_wrong": 480 + }, + { + "budget": 0.12, + "threshold": -1.5916669368743896, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.7769344296858528, + "rescued": 541, + "harmed": 162, + "missed_rescue": 1423, + "avoided_harm": 1009, + "selected_both_correct": 492, + "selected_both_wrong": 512 + }, + { + "budget": 0.13, + "threshold": -1.7836966514587402, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.7791130789233256, + "rescued": 584, + "harmed": 174, + "missed_rescue": 1380, + "avoided_harm": 997, + "selected_both_correct": 543, + "selected_both_wrong": 548 + }, + { + "budget": 0.14, + "threshold": -1.9795069694519043, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.781221449153138, + "rescued": 632, + "harmed": 192, + "missed_rescue": 1332, + "avoided_harm": 979, + "selected_both_correct": 597, + "selected_both_wrong": 571 + }, + { + "budget": 0.15, + "threshold": -2.1732606887817383, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.7838920514442336, + "rescued": 678, + "harmed": 200, + "missed_rescue": 1286, + "avoided_harm": 971, + "selected_both_correct": 654, + "selected_both_wrong": 602 + }, + { + "budget": 0.16, + "threshold": -2.3698818683624268, + "operator": "gt", + "actual_fraction": 0.15988474242743692, + "metric": 0.786351816712348, + "rescued": 724, + "harmed": 211, + "missed_rescue": 1240, + "avoided_harm": 960, + "selected_both_correct": 713, + "selected_both_wrong": 627 + }, + { + "budget": 0.17, + "threshold": -2.5250070095062256, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.7876168388502355, + "rescued": 756, + "harmed": 225, + "missed_rescue": 1208, + "avoided_harm": 946, + "selected_both_correct": 785, + "selected_both_wrong": 652 + }, + { + "budget": 0.18, + "threshold": -2.659658908843994, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.7886710239651417, + "rescued": 793, + "harmed": 247, + "missed_rescue": 1171, + "avoided_harm": 924, + "selected_both_correct": 845, + "selected_both_wrong": 676 + }, + { + "budget": 0.19, + "threshold": -2.7833452224731445, + "operator": "ge", + "actual_fraction": 0.18996415770609318, + "metric": 0.7901468831260102, + "rescued": 833, + "harmed": 266, + "missed_rescue": 1131, + "avoided_harm": 905, + "selected_both_correct": 904, + "selected_both_wrong": 700 + }, + { + "budget": 0.2, + "threshold": -2.928896903991699, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.7914119052638977, + "rescued": 870, + "harmed": 285, + "missed_rescue": 1094, + "avoided_harm": 886, + "selected_both_correct": 974, + "selected_both_wrong": 716 + }, + { + "budget": 0.21, + "threshold": -3.0729546546936035, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.7926066483941246, + "rescued": 905, + "harmed": 303, + "missed_rescue": 1059, + "avoided_harm": 868, + "selected_both_correct": 1051, + "selected_both_wrong": 729 + }, + { + "budget": 0.22, + "threshold": -3.207036018371582, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.792817485417106, + "rescued": 925, + "harmed": 320, + "missed_rescue": 1039, + "avoided_harm": 851, + "selected_both_correct": 1140, + "selected_both_wrong": 745 + }, + { + "budget": 0.23, + "threshold": -3.3137195110321045, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.794363623585635, + "rescued": 957, + "harmed": 330, + "missed_rescue": 1007, + "avoided_harm": 841, + "selected_both_correct": 1218, + "selected_both_wrong": 767 + }, + { + "budget": 0.24, + "threshold": -3.4117960929870605, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.7959800407618245, + "rescued": 989, + "harmed": 339, + "missed_rescue": 975, + "avoided_harm": 832, + "selected_both_correct": 1299, + "selected_both_wrong": 787 + }, + { + "budget": 0.25, + "threshold": -3.505338191986084, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.7963314358001266, + "rescued": 1018, + "harmed": 363, + "missed_rescue": 946, + "avoided_harm": 808, + "selected_both_correct": 1374, + "selected_both_wrong": 802 + }, + { + "budget": 0.26, + "threshold": -3.594294548034668, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.7976667369456744, + "rescued": 1044, + "harmed": 370, + "missed_rescue": 920, + "avoided_harm": 801, + "selected_both_correct": 1463, + "selected_both_wrong": 822 + }, + { + "budget": 0.27, + "threshold": -3.6820778846740723, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.7978072949609951, + "rescued": 1066, + "harmed": 390, + "missed_rescue": 898, + "avoided_harm": 781, + "selected_both_correct": 1552, + "selected_both_wrong": 833 + }, + { + "budget": 0.28, + "threshold": -3.7688918113708496, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.7990723170988826, + "rescued": 1096, + "harmed": 402, + "missed_rescue": 868, + "avoided_harm": 769, + "selected_both_correct": 1637, + "selected_both_wrong": 849 + }, + { + "budget": 0.29, + "threshold": -3.8456528186798096, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.7996345491601659, + "rescued": 1115, + "harmed": 413, + "missed_rescue": 849, + "avoided_harm": 758, + "selected_both_correct": 1736, + "selected_both_wrong": 862 + }, + { + "budget": 0.3, + "threshold": -3.9210715293884277, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.799845386183147, + "rescued": 1134, + "harmed": 429, + "missed_rescue": 830, + "avoided_harm": 742, + "selected_both_correct": 1830, + "selected_both_wrong": 875 + }, + { + "budget": 0.31, + "threshold": -4.0025529861450195, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.7997751071754867, + "rescued": 1147, + "harmed": 443, + "missed_rescue": 817, + "avoided_harm": 728, + "selected_both_correct": 1933, + "selected_both_wrong": 887 + }, + { + "budget": 0.32, + "threshold": -4.074873924255371, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.8008995712980532, + "rescued": 1169, + "harmed": 449, + "missed_rescue": 795, + "avoided_harm": 722, + "selected_both_correct": 2035, + "selected_both_wrong": 900 + }, + { + "budget": 0.33, + "threshold": -4.144405841827393, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.8016726403823178, + "rescued": 1192, + "harmed": 461, + "missed_rescue": 772, + "avoided_harm": 710, + "selected_both_correct": 2134, + "selected_both_wrong": 908 + }, + { + "budget": 0.34, + "threshold": -4.213827610015869, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.8027971045048844, + "rescued": 1215, + "harmed": 468, + "missed_rescue": 749, + "avoided_harm": 703, + "selected_both_correct": 2231, + "selected_both_wrong": 923 + }, + { + "budget": 0.35, + "threshold": -4.29035758972168, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.8037810106121301, + "rescued": 1235, + "harmed": 474, + "missed_rescue": 729, + "avoided_harm": 697, + "selected_both_correct": 2335, + "selected_both_wrong": 936 + }, + { + "budget": 0.36, + "threshold": -4.3551530838012695, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.8041324056504322, + "rescued": 1253, + "harmed": 487, + "missed_rescue": 711, + "avoided_harm": 684, + "selected_both_correct": 2435, + "selected_both_wrong": 947 + }, + { + "budget": 0.37, + "threshold": -4.414309978485107, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.8043432426734135, + "rescued": 1272, + "harmed": 503, + "missed_rescue": 692, + "avoided_harm": 668, + "selected_both_correct": 2530, + "selected_both_wrong": 959 + }, + { + "budget": 0.38, + "threshold": -4.483922481536865, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.8037810106121301, + "rescued": 1285, + "harmed": 524, + "missed_rescue": 679, + "avoided_harm": 647, + "selected_both_correct": 2631, + "selected_both_wrong": 967 + }, + { + "budget": 0.39, + "threshold": -4.539813041687012, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.8041324056504322, + "rescued": 1300, + "harmed": 534, + "missed_rescue": 664, + "avoided_harm": 637, + "selected_both_correct": 2738, + "selected_both_wrong": 977 + }, + { + "budget": 0.4, + "threshold": -4.609446048736572, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.8051865907653384, + "rescued": 1323, + "harmed": 542, + "missed_rescue": 641, + "avoided_harm": 629, + "selected_both_correct": 2839, + "selected_both_wrong": 987 + }, + { + "budget": 0.41, + "threshold": -4.663720607757568, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.8053271487806593, + "rescued": 1336, + "harmed": 553, + "missed_rescue": 628, + "avoided_harm": 618, + "selected_both_correct": 2941, + "selected_both_wrong": 1003 + }, + { + "budget": 0.42, + "threshold": -4.7261199951171875, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.8050460327500175, + "rescued": 1348, + "harmed": 569, + "missed_rescue": 616, + "avoided_harm": 602, + "selected_both_correct": 3045, + "selected_both_wrong": 1014 + }, + { + "budget": 0.43, + "threshold": -4.786887168884277, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.805116311757678, + "rescued": 1357, + "harmed": 577, + "missed_rescue": 607, + "avoided_harm": 594, + "selected_both_correct": 3159, + "selected_both_wrong": 1025 + }, + { + "budget": 0.44, + "threshold": -4.843598365783691, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.8055379858036404, + "rescued": 1371, + "harmed": 585, + "missed_rescue": 593, + "avoided_harm": 586, + "selected_both_correct": 3266, + "selected_both_wrong": 1038 + }, + { + "budget": 0.45, + "threshold": -4.890982151031494, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.8057488228266216, + "rescued": 1383, + "harmed": 594, + "missed_rescue": 581, + "avoided_harm": 577, + "selected_both_correct": 3378, + "selected_both_wrong": 1048 + }, + { + "budget": 0.46, + "threshold": -4.9397172927856445, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.8056082648113009, + "rescued": 1392, + "harmed": 605, + "missed_rescue": 572, + "avoided_harm": 566, + "selected_both_correct": 3488, + "selected_both_wrong": 1060 + }, + { + "budget": 0.47, + "threshold": -4.991722583770752, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.8059596598496029, + "rescued": 1407, + "harmed": 615, + "missed_rescue": 557, + "avoided_harm": 556, + "selected_both_correct": 3593, + "selected_both_wrong": 1072 + }, + { + "budget": 0.48, + "threshold": -5.048281669616699, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.8059596598496029, + "rescued": 1420, + "harmed": 628, + "missed_rescue": 544, + "avoided_harm": 543, + "selected_both_correct": 3697, + "selected_both_wrong": 1084 + }, + { + "budget": 0.49, + "threshold": -5.091976165771484, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.8063813338955654, + "rescued": 1435, + "harmed": 637, + "missed_rescue": 529, + "avoided_harm": 534, + "selected_both_correct": 3803, + "selected_both_wrong": 1097 + }, + { + "budget": 0.5, + "threshold": -5.137824535369873, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.8070841239721696, + "rescued": 1449, + "harmed": 641, + "missed_rescue": 515, + "avoided_harm": 530, + "selected_both_correct": 3913, + "selected_both_wrong": 1111 + }, + { + "budget": 0.51, + "threshold": -5.183772563934326, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.8065921709185466, + "rescued": 1454, + "harmed": 653, + "missed_rescue": 510, + "avoided_harm": 518, + "selected_both_correct": 4024, + "selected_both_wrong": 1125 + }, + { + "budget": 0.52, + "threshold": -5.225483417510986, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.8068732869491883, + "rescued": 1465, + "harmed": 660, + "missed_rescue": 499, + "avoided_harm": 511, + "selected_both_correct": 4140, + "selected_both_wrong": 1134 + }, + { + "budget": 0.53, + "threshold": -5.285849571228027, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.8071544029798299, + "rescued": 1479, + "harmed": 670, + "missed_rescue": 485, + "avoided_harm": 501, + "selected_both_correct": 4252, + "selected_both_wrong": 1140 + }, + { + "budget": 0.54, + "threshold": -5.33848237991333, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.807857193056434, + "rescued": 1493, + "harmed": 674, + "missed_rescue": 471, + "avoided_harm": 497, + "selected_both_correct": 4367, + "selected_both_wrong": 1149 + }, + { + "budget": 0.55, + "threshold": -5.383656978607178, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.8079977510717549, + "rescued": 1506, + "harmed": 685, + "missed_rescue": 458, + "avoided_harm": 486, + "selected_both_correct": 4474, + "selected_both_wrong": 1160 + }, + { + "budget": 0.56, + "threshold": -5.435944080352783, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.8081383090870757, + "rescued": 1513, + "harmed": 690, + "missed_rescue": 451, + "avoided_harm": 481, + "selected_both_correct": 4599, + "selected_both_wrong": 1166 + }, + { + "budget": 0.57, + "threshold": -5.478402614593506, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.8079274720640944, + "rescued": 1523, + "harmed": 703, + "missed_rescue": 441, + "avoided_harm": 468, + "selected_both_correct": 4705, + "selected_both_wrong": 1179 + }, + { + "budget": 0.58, + "threshold": -5.5247578620910645, + "operator": "gt", + "actual_fraction": 0.5798720922060581, + "metric": 0.8079274720640944, + "rescued": 1532, + "harmed": 712, + "missed_rescue": 432, + "avoided_harm": 459, + "selected_both_correct": 4819, + "selected_both_wrong": 1188 + }, + { + "budget": 0.59, + "threshold": -5.566192626953125, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.8080680300794153, + "rescued": 1543, + "harmed": 721, + "missed_rescue": 421, + "avoided_harm": 450, + "selected_both_correct": 4933, + "selected_both_wrong": 1198 + }, + { + "budget": 0.6, + "threshold": -5.613105773925781, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.807857193056434, + "rescued": 1551, + "harmed": 732, + "missed_rescue": 413, + "avoided_harm": 439, + "selected_both_correct": 5051, + "selected_both_wrong": 1203 + }, + { + "budget": 0.61, + "threshold": -5.660335540771484, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.8075057980181319, + "rescued": 1557, + "harmed": 743, + "missed_rescue": 407, + "avoided_harm": 428, + "selected_both_correct": 5166, + "selected_both_wrong": 1213 + }, + { + "budget": 0.62, + "threshold": -5.713016986846924, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.8070841239721696, + "rescued": 1560, + "harmed": 752, + "missed_rescue": 404, + "avoided_harm": 419, + "selected_both_correct": 5282, + "selected_both_wrong": 1227 + }, + { + "budget": 0.63, + "threshold": -5.765469074249268, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.8065921709185466, + "rescued": 1568, + "harmed": 767, + "missed_rescue": 396, + "avoided_harm": 404, + "selected_both_correct": 5389, + "selected_both_wrong": 1240 + }, + { + "budget": 0.64, + "threshold": -5.808561325073242, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.8065218919108862, + "rescued": 1579, + "harmed": 779, + "missed_rescue": 385, + "avoided_harm": 392, + "selected_both_correct": 5501, + "selected_both_wrong": 1247 + }, + { + "budget": 0.65, + "threshold": -5.852151870727539, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.8058893808419425, + "rescued": 1584, + "harmed": 793, + "missed_rescue": 380, + "avoided_harm": 378, + "selected_both_correct": 5618, + "selected_both_wrong": 1253 + }, + { + "budget": 0.66, + "threshold": -5.899008274078369, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.8056785438189613, + "rescued": 1593, + "harmed": 805, + "missed_rescue": 371, + "avoided_harm": 366, + "selected_both_correct": 5729, + "selected_both_wrong": 1264 + }, + { + "budget": 0.67, + "threshold": -5.944814682006836, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.8057488228266216, + "rescued": 1603, + "harmed": 814, + "missed_rescue": 361, + "avoided_harm": 357, + "selected_both_correct": 5849, + "selected_both_wrong": 1267 + }, + { + "budget": 0.68, + "threshold": -5.992681980133057, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.8060299388572634, + "rescued": 1620, + "harmed": 827, + "missed_rescue": 344, + "avoided_harm": 344, + "selected_both_correct": 5954, + "selected_both_wrong": 1274 + }, + { + "budget": 0.69, + "threshold": -6.03839635848999, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.8056082648113009, + "rescued": 1629, + "harmed": 842, + "missed_rescue": 335, + "avoided_harm": 329, + "selected_both_correct": 6066, + "selected_both_wrong": 1281 + }, + { + "budget": 0.7, + "threshold": -6.091725826263428, + "operator": "ge", + "actual_fraction": 0.6999789162977019, + "metric": 0.8055379858036404, + "rescued": 1635, + "harmed": 849, + "missed_rescue": 329, + "avoided_harm": 322, + "selected_both_correct": 6187, + "selected_both_wrong": 1289 + }, + { + "budget": 0.71, + "threshold": -6.139003276824951, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.8053974277883196, + "rescued": 1644, + "harmed": 860, + "missed_rescue": 320, + "avoided_harm": 311, + "selected_both_correct": 6302, + "selected_both_wrong": 1296 + }, + { + "budget": 0.72, + "threshold": -6.183269500732422, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.80546770679598, + "rescued": 1655, + "harmed": 870, + "missed_rescue": 309, + "avoided_harm": 301, + "selected_both_correct": 6417, + "selected_both_wrong": 1302 + }, + { + "budget": 0.73, + "threshold": -6.240170955657959, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.8055379858036404, + "rescued": 1664, + "harmed": 878, + "missed_rescue": 300, + "avoided_harm": 293, + "selected_both_correct": 6531, + "selected_both_wrong": 1314 + }, + { + "budget": 0.74, + "threshold": -6.2856950759887695, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.8058191018342821, + "rescued": 1676, + "harmed": 886, + "missed_rescue": 288, + "avoided_harm": 285, + "selected_both_correct": 6650, + "selected_both_wrong": 1317 + }, + { + "budget": 0.75, + "threshold": -6.338753700256348, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.8057488228266216, + "rescued": 1684, + "harmed": 895, + "missed_rescue": 280, + "avoided_harm": 276, + "selected_both_correct": 6764, + "selected_both_wrong": 1328 + }, + { + "budget": 0.76, + "threshold": -6.385316848754883, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.8058893808419425, + "rescued": 1695, + "harmed": 904, + "missed_rescue": 269, + "avoided_harm": 267, + "selected_both_correct": 6872, + "selected_both_wrong": 1343 + }, + { + "budget": 0.77, + "threshold": -6.438819408416748, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.8061704968725841, + "rescued": 1707, + "harmed": 912, + "missed_rescue": 257, + "avoided_harm": 259, + "selected_both_correct": 6987, + "selected_both_wrong": 1350 + }, + { + "budget": 0.78, + "threshold": -6.489089012145996, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.8062407758802446, + "rescued": 1719, + "harmed": 923, + "missed_rescue": 245, + "avoided_harm": 248, + "selected_both_correct": 7096, + "selected_both_wrong": 1360 + }, + { + "budget": 0.79, + "threshold": -6.554056644439697, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.8066624499262071, + "rescued": 1729, + "harmed": 927, + "missed_rescue": 235, + "avoided_harm": 244, + "selected_both_correct": 7218, + "selected_both_wrong": 1366 + }, + { + "budget": 0.8, + "threshold": -6.614213943481445, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.8068030079415278, + "rescued": 1740, + "harmed": 936, + "missed_rescue": 224, + "avoided_harm": 235, + "selected_both_correct": 7333, + "selected_both_wrong": 1374 + }, + { + "budget": 0.81, + "threshold": -6.680136680603027, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.8072949609951507, + "rescued": 1756, + "harmed": 945, + "missed_rescue": 208, + "avoided_harm": 226, + "selected_both_correct": 7439, + "selected_both_wrong": 1385 + }, + { + "budget": 0.82, + "threshold": -6.735454559326172, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.8072246819874903, + "rescued": 1761, + "harmed": 951, + "missed_rescue": 203, + "avoided_harm": 220, + "selected_both_correct": 7561, + "selected_both_wrong": 1394 + }, + { + "budget": 0.83, + "threshold": -6.790655136108398, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.8075760770257924, + "rescued": 1770, + "harmed": 955, + "missed_rescue": 194, + "avoided_harm": 216, + "selected_both_correct": 7684, + "selected_both_wrong": 1401 + }, + { + "budget": 0.84, + "threshold": -6.856695175170898, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.8075760770257924, + "rescued": 1783, + "harmed": 968, + "missed_rescue": 181, + "avoided_harm": 203, + "selected_both_correct": 7793, + "selected_both_wrong": 1408 + }, + { + "budget": 0.85, + "threshold": -6.932329177856445, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.8075760770257924, + "rescued": 1792, + "harmed": 977, + "missed_rescue": 172, + "avoided_harm": 194, + "selected_both_correct": 7906, + "selected_both_wrong": 1419 + }, + { + "budget": 0.86, + "threshold": -6.999267578125, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.8077166350411132, + "rescued": 1799, + "harmed": 982, + "missed_rescue": 165, + "avoided_harm": 189, + "selected_both_correct": 8025, + "selected_both_wrong": 1430 + }, + { + "budget": 0.87, + "threshold": -7.081692695617676, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.8077166350411132, + "rescued": 1807, + "harmed": 990, + "missed_rescue": 157, + "avoided_harm": 181, + "selected_both_correct": 8145, + "selected_both_wrong": 1437 + }, + { + "budget": 0.88, + "threshold": -7.155144214630127, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.8077869140487737, + "rescued": 1819, + "harmed": 1001, + "missed_rescue": 145, + "avoided_harm": 170, + "selected_both_correct": 8259, + "selected_both_wrong": 1442 + }, + { + "budget": 0.89, + "threshold": -7.227406978607178, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.8076463560334528, + "rescued": 1829, + "harmed": 1013, + "missed_rescue": 135, + "avoided_harm": 158, + "selected_both_correct": 8365, + "selected_both_wrong": 1456 + }, + { + "budget": 0.9, + "threshold": -7.308749198913574, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.8070138449645091, + "rescued": 1837, + "harmed": 1030, + "missed_rescue": 127, + "avoided_harm": 141, + "selected_both_correct": 8472, + "selected_both_wrong": 1467 + }, + { + "budget": 0.91, + "threshold": -7.405246257781982, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.8070138449645091, + "rescued": 1848, + "harmed": 1041, + "missed_rescue": 116, + "avoided_harm": 130, + "selected_both_correct": 8582, + "selected_both_wrong": 1477 + }, + { + "budget": 0.92, + "threshold": -7.509663105010986, + "operator": "gt", + "actual_fraction": 0.9198819312671305, + "metric": 0.8070138449645091, + "rescued": 1856, + "harmed": 1049, + "missed_rescue": 108, + "avoided_harm": 122, + "selected_both_correct": 8695, + "selected_both_wrong": 1489 + }, + { + "budget": 0.93, + "threshold": -7.625953674316406, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.8068030079415278, + "rescued": 1865, + "harmed": 1061, + "missed_rescue": 99, + "avoided_harm": 110, + "selected_both_correct": 8805, + "selected_both_wrong": 1501 + }, + { + "budget": 0.94, + "threshold": -7.762190818786621, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.8068030079415278, + "rescued": 1878, + "harmed": 1074, + "missed_rescue": 86, + "avoided_harm": 97, + "selected_both_correct": 8911, + "selected_both_wrong": 1512 + }, + { + "budget": 0.95, + "threshold": -7.891505241394043, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.8068732869491883, + "rescued": 1894, + "harmed": 1089, + "missed_rescue": 70, + "avoided_harm": 82, + "selected_both_correct": 9012, + "selected_both_wrong": 1522 + }, + { + "budget": 0.96, + "threshold": -8.049057960510254, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.8068732869491883, + "rescued": 1913, + "harmed": 1108, + "missed_rescue": 51, + "avoided_harm": 63, + "selected_both_correct": 9108, + "selected_both_wrong": 1530 + }, + { + "budget": 0.97, + "threshold": -8.248269081115723, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.8071544029798299, + "rescued": 1927, + "harmed": 1118, + "missed_rescue": 37, + "avoided_harm": 53, + "selected_both_correct": 9215, + "selected_both_wrong": 1542 + }, + { + "budget": 0.98, + "threshold": -8.543853759765625, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.8068030079415278, + "rescued": 1939, + "harmed": 1135, + "missed_rescue": 25, + "avoided_harm": 36, + "selected_both_correct": 9310, + "selected_both_wrong": 1560 + }, + { + "budget": 0.99, + "threshold": -8.96818733215332, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.8065218919108862, + "rescued": 1948, + "harmed": 1148, + "missed_rescue": 16, + "avoided_harm": 23, + "selected_both_correct": 9420, + "selected_both_wrong": 1570 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8060299388572634, + "rescued": 1964, + "harmed": 1171, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9505, + "selected_both_wrong": 1589 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.6695119530385957, + "rescued": 0, + "harmed": 0, + "missed_rescue": 5423, + "avoided_harm": 2771, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 5.873929023742676, + "operator": "ge", + "actual_fraction": 0.04701812097666109, + "metric": 0.6878314380512152, + "rescued": 655, + "harmed": 9, + "missed_rescue": 4768, + "avoided_harm": 2762, + "selected_both_correct": 18, + "selected_both_wrong": 976 + }, + { + "budget": 0.02, + "threshold": 3.9572336673736572, + "operator": "ge", + "actual_fraction": 0.06706746448118424, + "metric": 0.6980404389870403, + "rescued": 1022, + "harmed": 16, + "missed_rescue": 4401, + "avoided_harm": 2755, + "selected_both_correct": 64, + "selected_both_wrong": 1263 + }, + { + "budget": 0.03, + "threshold": 2.499727725982666, + "operator": "ge", + "actual_fraction": 0.08720188299350594, + "metric": 0.7065195814309616, + "rescued": 1340, + "harmed": 35, + "missed_rescue": 4083, + "avoided_harm": 2736, + "selected_both_correct": 157, + "selected_both_wrong": 1543 + }, + { + "budget": 0.04, + "threshold": 1.2223308086395264, + "operator": "ge", + "actual_fraction": 0.1055780846779911, + "metric": 0.712389756969061, + "rescued": 1582, + "harmed": 70, + "missed_rescue": 3841, + "avoided_harm": 2701, + "selected_both_correct": 311, + "selected_both_wrong": 1760 + }, + { + "budget": 0.05, + "threshold": 0.36731216311454773, + "operator": "ge", + "actual_fraction": 0.11998411933187761, + "metric": 0.7164449990074583, + "rescued": 1757, + "harmed": 102, + "missed_rescue": 3666, + "avoided_harm": 2669, + "selected_both_correct": 475, + "selected_both_wrong": 1897 + }, + { + "budget": 0.06, + "threshold": -0.14144867658615112, + "operator": "ge", + "actual_fraction": 0.13243342880639764, + "metric": 0.7199047159912656, + "rescued": 1911, + "harmed": 134, + "missed_rescue": 3512, + "avoided_harm": 2637, + "selected_both_correct": 596, + "selected_both_wrong": 2029 + }, + { + "budget": 0.07, + "threshold": -0.42117512226104736, + "operator": "ge", + "actual_fraction": 0.14235884638289426, + "metric": 0.7222017412018262, + "rescued": 2025, + "harmed": 167, + "missed_rescue": 3398, + "avoided_harm": 2604, + "selected_both_correct": 705, + "selected_both_wrong": 2123 + }, + { + "budget": 0.08, + "threshold": -0.6775990128517151, + "operator": "ge", + "actual_fraction": 0.1511215721861441, + "metric": 0.7239032413577972, + "rescued": 2118, + "harmed": 200, + "missed_rescue": 3305, + "avoided_harm": 2571, + "selected_both_correct": 789, + "selected_both_wrong": 2222 + }, + { + "budget": 0.09, + "threshold": -0.9111056327819824, + "operator": "ge", + "actual_fraction": 0.15957235629413266, + "metric": 0.7263136999120892, + "rescued": 2226, + "harmed": 223, + "missed_rescue": 3197, + "avoided_harm": 2548, + "selected_both_correct": 857, + "selected_both_wrong": 2321 + }, + { + "budget": 0.1, + "threshold": -1.1332147121429443, + "operator": "ge", + "actual_fraction": 0.16822164875365114, + "metric": 0.7290077418257097, + "rescued": 2339, + "harmed": 241, + "missed_rescue": 3084, + "avoided_harm": 2530, + "selected_both_correct": 914, + "selected_both_wrong": 2438 + }, + { + "budget": 0.11, + "threshold": -1.3457791805267334, + "operator": "ge", + "actual_fraction": 0.17695601622096815, + "metric": 0.7316450670674645, + "rescued": 2454, + "harmed": 263, + "missed_rescue": 2969, + "avoided_harm": 2508, + "selected_both_correct": 997, + "selected_both_wrong": 2526 + }, + { + "budget": 0.12, + "threshold": -1.5916669368743896, + "operator": "ge", + "actual_fraction": 0.18855457561750277, + "metric": 0.7345943340044806, + "rescued": 2587, + "harmed": 292, + "missed_rescue": 2836, + "avoided_harm": 2479, + "selected_both_correct": 1103, + "selected_both_wrong": 2667 + }, + { + "budget": 0.13, + "threshold": -1.7836966514587402, + "operator": "ge", + "actual_fraction": 0.19751580977228256, + "metric": 0.7366361341916456, + "rescued": 2685, + "harmed": 318, + "missed_rescue": 2738, + "avoided_harm": 2453, + "selected_both_correct": 1187, + "selected_both_wrong": 2775 + }, + { + "budget": 0.14, + "threshold": -1.9795069694519043, + "operator": "ge", + "actual_fraction": 0.20664719394265946, + "metric": 0.7385361426991465, + "rescued": 2776, + "harmed": 342, + "missed_rescue": 2647, + "avoided_harm": 2429, + "selected_both_correct": 1271, + "selected_both_wrong": 2898 + }, + { + "budget": 0.15, + "threshold": -2.1732606887817383, + "operator": "ge", + "actual_fraction": 0.2172532115815444, + "metric": 0.7402376428551173, + "rescued": 2874, + "harmed": 380, + "missed_rescue": 2549, + "avoided_harm": 2391, + "selected_both_correct": 1386, + "selected_both_wrong": 3021 + }, + { + "budget": 0.16, + "threshold": -2.3698818683624268, + "operator": "gt", + "actual_fraction": 0.22717862915804102, + "metric": 0.7420242180188866, + "rescued": 2970, + "harmed": 413, + "missed_rescue": 2453, + "avoided_harm": 2358, + "selected_both_correct": 1489, + "selected_both_wrong": 3139 + }, + { + "budget": 0.17, + "threshold": -2.5250070095062256, + "operator": "ge", + "actual_fraction": 0.2361398633128208, + "metric": 0.7436122848311261, + "rescued": 3053, + "harmed": 440, + "missed_rescue": 2370, + "avoided_harm": 2331, + "selected_both_correct": 1581, + "selected_both_wrong": 3253 + }, + { + "budget": 0.18, + "threshold": -2.659658908843994, + "operator": "ge", + "actual_fraction": 0.24362646399909255, + "metric": 0.7448033349403057, + "rescued": 3124, + "harmed": 469, + "missed_rescue": 2299, + "avoided_harm": 2302, + "selected_both_correct": 1685, + "selected_both_wrong": 3313 + }, + { + "budget": 0.19, + "threshold": -2.7833452224731445, + "operator": "ge", + "actual_fraction": 0.2507727646541701, + "metric": 0.7456540850182911, + "rescued": 3179, + "harmed": 494, + "missed_rescue": 2244, + "avoided_harm": 2277, + "selected_both_correct": 1781, + "selected_both_wrong": 3389 + }, + { + "budget": 0.2, + "threshold": -2.928896903991699, + "operator": "ge", + "actual_fraction": 0.25942205711368854, + "metric": 0.7459943850494853, + "rescued": 3237, + "harmed": 540, + "missed_rescue": 2186, + "avoided_harm": 2231, + "selected_both_correct": 1913, + "selected_both_wrong": 3458 + }, + { + "budget": 0.21, + "threshold": -3.0729546546936035, + "operator": "ge", + "actual_fraction": 0.2690922496667896, + "metric": 0.746419760088478, + "rescued": 3296, + "harmed": 584, + "missed_rescue": 2127, + "avoided_harm": 2187, + "selected_both_correct": 2081, + "selected_both_wrong": 3528 + }, + { + "budget": 0.22, + "threshold": -3.207036018371582, + "operator": "ge", + "actual_fraction": 0.2781385588293679, + "metric": 0.7467600601196722, + "rescued": 3347, + "harmed": 623, + "missed_rescue": 2076, + "avoided_harm": 2148, + "selected_both_correct": 2230, + "selected_both_wrong": 3608 + }, + { + "budget": 0.23, + "threshold": -3.3137195110321045, + "operator": "ge", + "actual_fraction": 0.2868445679607521, + "metric": 0.7473555851742619, + "rescued": 3396, + "harmed": 651, + "missed_rescue": 2027, + "avoided_harm": 2120, + "selected_both_correct": 2391, + "selected_both_wrong": 3677 + }, + { + "budget": 0.24, + "threshold": -3.4117960929870605, + "operator": "ge", + "actual_fraction": 0.29430281031109096, + "metric": 0.7480645435725831, + "rescued": 3446, + "harmed": 676, + "missed_rescue": 1977, + "avoided_harm": 2095, + "selected_both_correct": 2535, + "selected_both_wrong": 3721 + }, + { + "budget": 0.25, + "threshold": -3.505338191986084, + "operator": "ge", + "actual_fraction": 0.30176105266142983, + "metric": 0.748092901908516, + "rescued": 3479, + "harmed": 708, + "missed_rescue": 1944, + "avoided_harm": 2063, + "selected_both_correct": 2673, + "selected_both_wrong": 3781 + }, + { + "budget": 0.26, + "threshold": -3.594294548034668, + "operator": "ge", + "actual_fraction": 0.30995661174602274, + "metric": 0.7486033519553073, + "rescued": 3520, + "harmed": 731, + "missed_rescue": 1903, + "avoided_harm": 2040, + "selected_both_correct": 2847, + "selected_both_wrong": 3832 + }, + { + "budget": 0.27, + "threshold": -3.6820778846740723, + "operator": "ge", + "actual_fraction": 0.3175850041119587, + "metric": 0.7494541020332927, + "rescued": 3569, + "harmed": 750, + "missed_rescue": 1854, + "avoided_harm": 2021, + "selected_both_correct": 3000, + "selected_both_wrong": 3880 + }, + { + "budget": 0.28, + "threshold": -3.7688918113708496, + "operator": "ge", + "actual_fraction": 0.3262342965714772, + "metric": 0.7501630604316138, + "rescued": 3623, + "harmed": 779, + "missed_rescue": 1800, + "avoided_harm": 1992, + "selected_both_correct": 3177, + "selected_both_wrong": 3925 + }, + { + "budget": 0.29, + "threshold": -3.8456528186798096, + "operator": "ge", + "actual_fraction": 0.3324731304767036, + "metric": 0.7505600771346738, + "rescued": 3655, + "harmed": 797, + "missed_rescue": 1768, + "avoided_harm": 1974, + "selected_both_correct": 3301, + "selected_both_wrong": 3971 + }, + { + "budget": 0.3, + "threshold": -3.9210715293884277, + "operator": "ge", + "actual_fraction": 0.3399313728270425, + "metric": 0.7509570938377336, + "rescued": 3698, + "harmed": 826, + "missed_rescue": 1725, + "avoided_harm": 1945, + "selected_both_correct": 3454, + "selected_both_wrong": 4009 + }, + { + "budget": 0.31, + "threshold": -4.0025529861450195, + "operator": "ge", + "actual_fraction": 0.34824036525536683, + "metric": 0.7511839605251964, + "rescued": 3732, + "harmed": 852, + "missed_rescue": 1691, + "avoided_harm": 1919, + "selected_both_correct": 3635, + "selected_both_wrong": 4061 + }, + { + "budget": 0.32, + "threshold": -4.074873924255371, + "operator": "ge", + "actual_fraction": 0.3557553242775714, + "metric": 0.751269035532995, + "rescued": 3761, + "harmed": 878, + "missed_rescue": 1662, + "avoided_harm": 1893, + "selected_both_correct": 3798, + "selected_both_wrong": 4108 + }, + { + "budget": 0.33, + "threshold": -4.144405841827393, + "operator": "ge", + "actual_fraction": 0.3627598332529847, + "metric": 0.7511272438533307, + "rescued": 3779, + "harmed": 901, + "missed_rescue": 1644, + "avoided_harm": 1870, + "selected_both_correct": 3962, + "selected_both_wrong": 4150 + }, + { + "budget": 0.34, + "threshold": -4.213827610015869, + "operator": "ge", + "actual_fraction": 0.37013300059552506, + "metric": 0.7512123188611293, + "rescued": 3806, + "harmed": 925, + "missed_rescue": 1617, + "avoided_harm": 1846, + "selected_both_correct": 4131, + "selected_both_wrong": 4190 + }, + { + "budget": 0.35, + "threshold": -4.29035758972168, + "operator": "ge", + "actual_fraction": 0.3783569180160508, + "metric": 0.751439185548592, + "rescued": 3839, + "harmed": 950, + "missed_rescue": 1584, + "avoided_harm": 1821, + "selected_both_correct": 4321, + "selected_both_wrong": 4232 + }, + { + "budget": 0.36, + "threshold": -4.3551530838012695, + "operator": "ge", + "actual_fraction": 0.3854748603351955, + "metric": 0.7517511272438533, + "rescued": 3869, + "harmed": 969, + "missed_rescue": 1554, + "avoided_harm": 1802, + "selected_both_correct": 4491, + "selected_both_wrong": 4264 + }, + { + "budget": 0.37, + "threshold": -4.414309978485107, + "operator": "ge", + "actual_fraction": 0.39279131100587017, + "metric": 0.752176502282846, + "rescued": 3898, + "harmed": 983, + "missed_rescue": 1525, + "avoided_harm": 1788, + "selected_both_correct": 4671, + "selected_both_wrong": 4299 + }, + { + "budget": 0.38, + "threshold": -4.483922481536865, + "operator": "ge", + "actual_fraction": 0.40013612001247767, + "metric": 0.7522332189547117, + "rescued": 3921, + "harmed": 1004, + "missed_rescue": 1502, + "avoided_harm": 1767, + "selected_both_correct": 4853, + "selected_both_wrong": 4332 + }, + { + "budget": 0.39, + "threshold": -4.539813041687012, + "operator": "ge", + "actual_fraction": 0.406573462269234, + "metric": 0.7525735189859059, + "rescued": 3947, + "harmed": 1018, + "missed_rescue": 1476, + "avoided_harm": 1753, + "selected_both_correct": 5011, + "selected_both_wrong": 4361 + }, + { + "budget": 0.4, + "threshold": -4.609446048736572, + "operator": "ge", + "actual_fraction": 0.4153929047443496, + "metric": 0.7528003856733687, + "rescued": 3980, + "harmed": 1043, + "missed_rescue": 1443, + "avoided_harm": 1728, + "selected_both_correct": 5214, + "selected_both_wrong": 4411 + }, + { + "budget": 0.41, + "threshold": -4.663720607757568, + "operator": "ge", + "actual_fraction": 0.42151830530584466, + "metric": 0.75311232736863, + "rescued": 4010, + "harmed": 1062, + "missed_rescue": 1413, + "avoided_harm": 1709, + "selected_both_correct": 5358, + "selected_both_wrong": 4434 + }, + { + "budget": 0.42, + "threshold": -4.7261199951171875, + "operator": "ge", + "actual_fraction": 0.42920341434364634, + "metric": 0.7535093440716899, + "rescued": 4045, + "harmed": 1083, + "missed_rescue": 1378, + "avoided_harm": 1688, + "selected_both_correct": 5538, + "selected_both_wrong": 4469 + }, + { + "budget": 0.43, + "threshold": -4.786887168884277, + "operator": "ge", + "actual_fraction": 0.4367750900377166, + "metric": 0.7537362107591526, + "rescued": 4070, + "harmed": 1100, + "missed_rescue": 1353, + "avoided_harm": 1671, + "selected_both_correct": 5721, + "selected_both_wrong": 4511 + }, + { + "budget": 0.44, + "threshold": -4.843598365783691, + "operator": "ge", + "actual_fraction": 0.4450273657941752, + "metric": 0.753679494087287, + "rescued": 4099, + "harmed": 1131, + "missed_rescue": 1324, + "avoided_harm": 1640, + "selected_both_correct": 5910, + "selected_both_wrong": 4553 + }, + { + "budget": 0.45, + "threshold": -4.890982151031494, + "operator": "ge", + "actual_fraction": 0.45143634971499874, + "metric": 0.7537929274310183, + "rescued": 4122, + "harmed": 1150, + "missed_rescue": 1301, + "avoided_harm": 1621, + "selected_both_correct": 6067, + "selected_both_wrong": 4580 + }, + { + "budget": 0.46, + "threshold": -4.9397172927856445, + "operator": "ge", + "actual_fraction": 0.4578736919717551, + "metric": 0.7536227774154213, + "rescued": 4144, + "harmed": 1178, + "missed_rescue": 1279, + "avoided_harm": 1593, + "selected_both_correct": 6218, + "selected_both_wrong": 4606 + }, + { + "budget": 0.47, + "threshold": -4.991722583770752, + "operator": "ge", + "actual_fraction": 0.4659274593766838, + "metric": 0.7537929274310183, + "rescued": 4172, + "harmed": 1200, + "missed_rescue": 1251, + "avoided_harm": 1571, + "selected_both_correct": 6425, + "selected_both_wrong": 4633 + }, + { + "budget": 0.48, + "threshold": -5.048281669616699, + "operator": "ge", + "actual_fraction": 0.4741797351331424, + "metric": 0.7538780024388169, + "rescued": 4202, + "harmed": 1227, + "missed_rescue": 1221, + "avoided_harm": 1544, + "selected_both_correct": 6628, + "selected_both_wrong": 4664 + }, + { + "budget": 0.49, + "threshold": -5.091976165771484, + "operator": "ge", + "actual_fraction": 0.4806454357258316, + "metric": 0.7537362107591526, + "rescued": 4222, + "harmed": 1252, + "missed_rescue": 1201, + "avoided_harm": 1519, + "selected_both_correct": 6781, + "selected_both_wrong": 4694 + }, + { + "budget": 0.5, + "threshold": -5.137824535369873, + "operator": "ge", + "actual_fraction": 0.487451436349715, + "metric": 0.7535093440716899, + "rescued": 4237, + "harmed": 1275, + "missed_rescue": 1186, + "avoided_harm": 1496, + "selected_both_correct": 6963, + "selected_both_wrong": 4714 + }, + { + "budget": 0.51, + "threshold": -5.183772563934326, + "operator": "ge", + "actual_fraction": 0.49482460369225534, + "metric": 0.7537645690950855, + "rescued": 4267, + "harmed": 1296, + "missed_rescue": 1156, + "avoided_harm": 1475, + "selected_both_correct": 7140, + "selected_both_wrong": 4746 + }, + { + "budget": 0.52, + "threshold": -5.225483417510986, + "operator": "ge", + "actual_fraction": 0.501573887644273, + "metric": 0.7535660607435556, + "rescued": 4282, + "harmed": 1318, + "missed_rescue": 1141, + "avoided_harm": 1453, + "selected_both_correct": 7308, + "selected_both_wrong": 4779 + }, + { + "budget": 0.53, + "threshold": -5.285849571228027, + "operator": "ge", + "actual_fraction": 0.5105351217990528, + "metric": 0.7532257607123614, + "rescued": 4298, + "harmed": 1346, + "missed_rescue": 1125, + "avoided_harm": 1425, + "selected_both_correct": 7555, + "selected_both_wrong": 4804 + }, + { + "budget": 0.54, + "threshold": -5.33848237991333, + "operator": "ge", + "actual_fraction": 0.519241130930437, + "metric": 0.7530556106967643, + "rescued": 4319, + "harmed": 1373, + "missed_rescue": 1104, + "avoided_harm": 1398, + "selected_both_correct": 7769, + "selected_both_wrong": 4849 + }, + { + "budget": 0.55, + "threshold": -5.383656978607178, + "operator": "ge", + "actual_fraction": 0.5268128066245072, + "metric": 0.7528571023452344, + "rescued": 4337, + "harmed": 1398, + "missed_rescue": 1086, + "avoided_harm": 1373, + "selected_both_correct": 7963, + "selected_both_wrong": 4879 + }, + { + "budget": 0.56, + "threshold": -5.435944080352783, + "operator": "ge", + "actual_fraction": 0.5355755324277571, + "metric": 0.7528287440093016, + "rescued": 4361, + "harmed": 1423, + "missed_rescue": 1062, + "avoided_harm": 1348, + "selected_both_correct": 8197, + "selected_both_wrong": 4905 + }, + { + "budget": 0.57, + "threshold": -5.478402614593506, + "operator": "ge", + "actual_fraction": 0.5419561580126478, + "metric": 0.753282477384227, + "rescued": 4390, + "harmed": 1436, + "missed_rescue": 1033, + "avoided_harm": 1335, + "selected_both_correct": 8356, + "selected_both_wrong": 4929 + }, + { + "budget": 0.58, + "threshold": -5.5247578620910645, + "operator": "gt", + "actual_fraction": 0.5495845503785838, + "metric": 0.7528287440093016, + "rescued": 4406, + "harmed": 1468, + "missed_rescue": 1017, + "avoided_harm": 1303, + "selected_both_correct": 8545, + "selected_both_wrong": 4961 + }, + { + "budget": 0.59, + "threshold": -5.566192626953125, + "operator": "ge", + "actual_fraction": 0.5562771176587358, + "metric": 0.7527153106655702, + "rescued": 4430, + "harmed": 1496, + "missed_rescue": 993, + "avoided_harm": 1275, + "selected_both_correct": 8706, + "selected_both_wrong": 4984 + }, + { + "budget": 0.6, + "threshold": -5.613105773925781, + "operator": "ge", + "actual_fraction": 0.564047301704336, + "metric": 0.7526585939937045, + "rescued": 4452, + "harmed": 1520, + "missed_rescue": 971, + "avoided_harm": 1251, + "selected_both_correct": 8905, + "selected_both_wrong": 5013 + }, + { + "budget": 0.61, + "threshold": -5.660335540771484, + "operator": "ge", + "actual_fraction": 0.5725831608201231, + "metric": 0.7528287440093016, + "rescued": 4481, + "harmed": 1543, + "missed_rescue": 942, + "avoided_harm": 1228, + "selected_both_correct": 9128, + "selected_both_wrong": 5039 + }, + { + "budget": 0.62, + "threshold": -5.713016986846924, + "operator": "ge", + "actual_fraction": 0.5818846950060971, + "metric": 0.7528571023452344, + "rescued": 4511, + "harmed": 1572, + "missed_rescue": 912, + "avoided_harm": 1199, + "selected_both_correct": 9371, + "selected_both_wrong": 5065 + }, + { + "budget": 0.63, + "threshold": -5.765469074249268, + "operator": "ge", + "actual_fraction": 0.5903638374500184, + "metric": 0.7528571023452344, + "rescued": 4535, + "harmed": 1596, + "missed_rescue": 888, + "avoided_harm": 1175, + "selected_both_correct": 9596, + "selected_both_wrong": 5091 + }, + { + "budget": 0.64, + "threshold": -5.808561325073242, + "operator": "ge", + "actual_fraction": 0.5973683464254318, + "metric": 0.7526869523296373, + "rescued": 4548, + "harmed": 1615, + "missed_rescue": 875, + "avoided_harm": 1156, + "selected_both_correct": 9794, + "selected_both_wrong": 5108 + }, + { + "budget": 0.65, + "threshold": -5.852151870727539, + "operator": "ge", + "actual_fraction": 0.6042594220571137, + "metric": 0.7526018773218388, + "rescued": 4570, + "harmed": 1640, + "missed_rescue": 853, + "avoided_harm": 1131, + "selected_both_correct": 9969, + "selected_both_wrong": 5129 + }, + { + "budget": 0.66, + "threshold": -5.899008274078369, + "operator": "ge", + "actual_fraction": 0.6122848311261095, + "metric": 0.752545160649973, + "rescued": 4589, + "harmed": 1661, + "missed_rescue": 834, + "avoided_harm": 1110, + "selected_both_correct": 10189, + "selected_both_wrong": 5152 + }, + { + "budget": 0.67, + "threshold": -5.944814682006836, + "operator": "ge", + "actual_fraction": 0.6198848651561126, + "metric": 0.7522048606187789, + "rescued": 4603, + "harmed": 1687, + "missed_rescue": 820, + "avoided_harm": 1084, + "selected_both_correct": 10390, + "selected_both_wrong": 5179 + }, + { + "budget": 0.68, + "threshold": -5.992681980133057, + "operator": "ge", + "actual_fraction": 0.6280520659047727, + "metric": 0.752176502282846, + "rescued": 4622, + "harmed": 1707, + "missed_rescue": 801, + "avoided_harm": 1064, + "selected_both_correct": 10612, + "selected_both_wrong": 5206 + }, + { + "budget": 0.69, + "threshold": -6.03839635848999, + "operator": "ge", + "actual_fraction": 0.6363894166690298, + "metric": 0.7522048606187789, + "rescued": 4651, + "harmed": 1735, + "missed_rescue": 772, + "avoided_harm": 1036, + "selected_both_correct": 10825, + "selected_both_wrong": 5230 + }, + { + "budget": 0.7, + "threshold": -6.091725826263428, + "operator": "ge", + "actual_fraction": 0.6444999007458242, + "metric": 0.7521197856109804, + "rescued": 4673, + "harmed": 1760, + "missed_rescue": 750, + "avoided_harm": 1011, + "selected_both_correct": 11035, + "selected_both_wrong": 5259 + }, + { + "budget": 0.71, + "threshold": -6.139003276824951, + "operator": "ge", + "actual_fraction": 0.6527521765022829, + "metric": 0.7519212772594505, + "rescued": 4686, + "harmed": 1780, + "missed_rescue": 737, + "avoided_harm": 991, + "selected_both_correct": 11264, + "selected_both_wrong": 5288 + }, + { + "budget": 0.72, + "threshold": -6.183269500732422, + "operator": "ge", + "actual_fraction": 0.6601253438448231, + "metric": 0.7518362022516518, + "rescued": 4704, + "harmed": 1801, + "missed_rescue": 719, + "avoided_harm": 970, + "selected_both_correct": 11458, + "selected_both_wrong": 5315 + }, + { + "budget": 0.73, + "threshold": -6.240170955657959, + "operator": "ge", + "actual_fraction": 0.6697955363979242, + "metric": 0.7518362022516518, + "rescued": 4732, + "harmed": 1829, + "missed_rescue": 691, + "avoided_harm": 942, + "selected_both_correct": 11714, + "selected_both_wrong": 5344 + }, + { + "budget": 0.74, + "threshold": -6.2856950759887695, + "operator": "ge", + "actual_fraction": 0.6775373621075915, + "metric": 0.7518645605875848, + "rescued": 4754, + "harmed": 1850, + "missed_rescue": 669, + "avoided_harm": 921, + "selected_both_correct": 11918, + "selected_both_wrong": 5370 + }, + { + "budget": 0.75, + "threshold": -6.338753700256348, + "operator": "ge", + "actual_fraction": 0.6870657629810283, + "metric": 0.7512973938689278, + "rescued": 4769, + "harmed": 1885, + "missed_rescue": 654, + "avoided_harm": 886, + "selected_both_correct": 12181, + "selected_both_wrong": 5393 + }, + { + "budget": 0.76, + "threshold": -6.385316848754883, + "operator": "ge", + "actual_fraction": 0.6956299804327482, + "metric": 0.7514108272126592, + "rescued": 4786, + "harmed": 1898, + "missed_rescue": 637, + "avoided_harm": 873, + "selected_both_correct": 12430, + "selected_both_wrong": 5416 + }, + { + "budget": 0.77, + "threshold": -6.438819408416748, + "operator": "ge", + "actual_fraction": 0.7044494229078637, + "metric": 0.7514959022204577, + "rescued": 4805, + "harmed": 1914, + "missed_rescue": 618, + "avoided_harm": 857, + "selected_both_correct": 12668, + "selected_both_wrong": 5454 + }, + { + "budget": 0.78, + "threshold": -6.489089012145996, + "operator": "ge", + "actual_fraction": 0.7129852820236509, + "metric": 0.7510705271814649, + "rescued": 4813, + "harmed": 1937, + "missed_rescue": 610, + "avoided_harm": 834, + "selected_both_correct": 12915, + "selected_both_wrong": 5477 + }, + { + "budget": 0.79, + "threshold": -6.554056644439697, + "operator": "ge", + "actual_fraction": 0.7243853330686555, + "metric": 0.7506735104784051, + "rescued": 4834, + "harmed": 1972, + "missed_rescue": 589, + "avoided_harm": 799, + "selected_both_correct": 13238, + "selected_both_wrong": 5500 + }, + { + "budget": 0.8, + "threshold": -6.614213943481445, + "operator": "ge", + "actual_fraction": 0.7337152255905623, + "metric": 0.7507302271502708, + "rescued": 4858, + "harmed": 1994, + "missed_rescue": 565, + "avoided_harm": 777, + "selected_both_correct": 13488, + "selected_both_wrong": 5533 + }, + { + "budget": 0.81, + "threshold": -6.680136680603027, + "operator": "ge", + "actual_fraction": 0.744377959901313, + "metric": 0.7505033604628081, + "rescued": 4880, + "harmed": 2024, + "missed_rescue": 543, + "avoided_harm": 747, + "selected_both_correct": 13784, + "selected_both_wrong": 5561 + }, + { + "budget": 0.82, + "threshold": -6.735454559326172, + "operator": "ge", + "actual_fraction": 0.7522048606187789, + "metric": 0.7503048521112781, + "rescued": 4897, + "harmed": 2048, + "missed_rescue": 526, + "avoided_harm": 723, + "selected_both_correct": 13998, + "selected_both_wrong": 5582 + }, + { + "budget": 0.83, + "threshold": -6.790655136108398, + "operator": "ge", + "actual_fraction": 0.7607123613986331, + "metric": 0.7503332104472109, + "rescued": 4923, + "harmed": 2073, + "missed_rescue": 500, + "avoided_harm": 698, + "selected_both_correct": 14223, + "selected_both_wrong": 5606 + }, + { + "budget": 0.84, + "threshold": -6.856695175170898, + "operator": "ge", + "actual_fraction": 0.770779570654794, + "metric": 0.749964552080084, + "rescued": 4937, + "harmed": 2100, + "missed_rescue": 486, + "avoided_harm": 671, + "selected_both_correct": 14511, + "selected_both_wrong": 5632 + }, + { + "budget": 0.85, + "threshold": -6.932329177856445, + "operator": "ge", + "actual_fraction": 0.7827467884184556, + "metric": 0.749964552080084, + "rescued": 4965, + "harmed": 2128, + "missed_rescue": 458, + "avoided_harm": 643, + "selected_both_correct": 14849, + "selected_both_wrong": 5660 + }, + { + "budget": 0.86, + "threshold": -6.999267578125, + "operator": "ge", + "actual_fraction": 0.793466239401072, + "metric": 0.7498511187363526, + "rescued": 4987, + "harmed": 2154, + "missed_rescue": 436, + "avoided_harm": 617, + "selected_both_correct": 15148, + "selected_both_wrong": 5691 + }, + { + "budget": 0.87, + "threshold": -7.081692695617676, + "operator": "ge", + "actual_fraction": 0.8059439072115249, + "metric": 0.7499361937441511, + "rescued": 5015, + "harmed": 2179, + "missed_rescue": 408, + "avoided_harm": 592, + "selected_both_correct": 15496, + "selected_both_wrong": 5730 + }, + { + "budget": 0.88, + "threshold": -7.155144214630127, + "operator": "ge", + "actual_fraction": 0.8164932081785441, + "metric": 0.7497093270566884, + "rescued": 5038, + "harmed": 2210, + "missed_rescue": 385, + "avoided_harm": 561, + "selected_both_correct": 15779, + "selected_both_wrong": 5765 + }, + { + "budget": 0.89, + "threshold": -7.227406978607178, + "operator": "ge", + "actual_fraction": 0.8270425091455633, + "metric": 0.7494257436973598, + "rescued": 5061, + "harmed": 2243, + "missed_rescue": 362, + "avoided_harm": 528, + "selected_both_correct": 16073, + "selected_both_wrong": 5787 + }, + { + "budget": 0.9, + "threshold": -7.308749198913574, + "operator": "ge", + "actual_fraction": 0.8367410600345971, + "metric": 0.749397385361427, + "rescued": 5087, + "harmed": 2270, + "missed_rescue": 336, + "avoided_harm": 501, + "selected_both_correct": 16348, + "selected_both_wrong": 5801 + }, + { + "budget": 0.91, + "threshold": -7.405246257781982, + "operator": "ge", + "actual_fraction": 0.8491903695091172, + "metric": 0.74883021864277, + "rescued": 5113, + "harmed": 2316, + "missed_rescue": 310, + "avoided_harm": 455, + "selected_both_correct": 16682, + "selected_both_wrong": 5834 + }, + { + "budget": 0.92, + "threshold": -7.509663105010986, + "operator": "gt", + "actual_fraction": 0.8618098289992343, + "metric": 0.7487167852990386, + "rescued": 5142, + "harmed": 2349, + "missed_rescue": 281, + "avoided_harm": 422, + "selected_both_correct": 17025, + "selected_both_wrong": 5874 + }, + { + "budget": 0.93, + "threshold": -7.625953674316406, + "operator": "ge", + "actual_fraction": 0.8745710801690156, + "metric": 0.7486884269631058, + "rescued": 5170, + "harmed": 2378, + "missed_rescue": 253, + "avoided_harm": 393, + "selected_both_correct": 17383, + "selected_both_wrong": 5909 + }, + { + "budget": 0.94, + "threshold": -7.762190818786621, + "operator": "ge", + "actual_fraction": 0.8879845730652526, + "metric": 0.7486884269631058, + "rescued": 5198, + "harmed": 2406, + "missed_rescue": 225, + "avoided_harm": 365, + "selected_both_correct": 17763, + "selected_both_wrong": 5946 + }, + { + "budget": 0.95, + "threshold": -7.891505241394043, + "operator": "ge", + "actual_fraction": 0.9000085075007799, + "metric": 0.7484332019397102, + "rescued": 5216, + "harmed": 2433, + "missed_rescue": 207, + "avoided_harm": 338, + "selected_both_correct": 18108, + "selected_both_wrong": 5980 + }, + { + "budget": 0.96, + "threshold": -8.049057960510254, + "operator": "ge", + "actual_fraction": 0.9125996086549641, + "metric": 0.7477809602132547, + "rescued": 5244, + "harmed": 2484, + "missed_rescue": 179, + "avoided_harm": 287, + "selected_both_correct": 18442, + "selected_both_wrong": 6011 + }, + { + "budget": 0.97, + "threshold": -8.248269081115723, + "operator": "ge", + "actual_fraction": 0.9269489266369849, + "metric": 0.7476391685335905, + "rescued": 5273, + "harmed": 2518, + "missed_rescue": 150, + "avoided_harm": 253, + "selected_both_correct": 18844, + "selected_both_wrong": 6052 + }, + { + "budget": 0.98, + "threshold": -8.543853759765625, + "operator": "ge", + "actual_fraction": 0.9459773700479256, + "metric": 0.7472988685023962, + "rescued": 5315, + "harmed": 2572, + "missed_rescue": 108, + "avoided_harm": 199, + "selected_both_correct": 19374, + "selected_both_wrong": 6097 + }, + { + "budget": 0.99, + "threshold": -8.96818733215332, + "operator": "ge", + "actual_fraction": 0.9658282052009188, + "metric": 0.7472421518305306, + "rescued": 5353, + "harmed": 2612, + "missed_rescue": 70, + "avoided_harm": 159, + "selected_both_correct": 19948, + "selected_both_wrong": 6145 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.7447182599325072, + "rescued": 5423, + "harmed": 2771, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 20838, + "selected_both_wrong": 6231 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 9505, + "rescuable": 1964, + "harmful": 1171, + "both_wrong": 1589, + "unit": "sample", + "total": 14229 + }, + "test": { + "both_correct": 20838, + "rescuable": 5423, + "harmful": 2771, + "both_wrong": 6231, + "unit": "sample", + "total": 35263 + } + }, + "small": { + "validation": 0.7502986857825568, + "test": 0.6695119530385957 + }, + "large": { + "validation": 0.8060299388572634, + "test": 0.7447182599325072 + }, + "learned_minus_confidence_test": 0.02850012761251175, + "learned_minus_small_test": 0.08331679097070588, + "learned_minus_large_test": 0.008110484076794378 + }, + { + "key": [ + "nlvr2", + "nlvr2", + "MiniCPM" + ], + "expert_display": "BEiT3", + "run": "shared_12_nlvr2_MiniCPM", + "validation_rows": 6982, + "test_rows": 13937, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/nlvr2/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 86, + "validation_area_0_25": 0.8284302492122602, + "validation_point": { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8587797192781438, + "metric": 0.9004583213978803, + "rescued": 1073, + "harmed": 359, + "missed_rescue": 14, + "avoided_harm": 24, + "selected_both_correct": 4254, + "selected_both_wrong": 310 + }, + "test_point": { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8580038745784602, + "metric": 0.9077276314845376, + "rescued": 2191, + "harmed": 690, + "missed_rescue": 34, + "avoided_harm": 39, + "selected_both_correct": 8533, + "selected_both_wrong": 544 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7981953594958464, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1087, + "avoided_harm": 383, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5245773792266846, + "operator": "gt", + "actual_fraction": 0.0097393297049556, + "metric": 0.8009166427957606, + "rescued": 26, + "harmed": 7, + "missed_rescue": 1061, + "avoided_harm": 376, + "selected_both_correct": 33, + "selected_both_wrong": 2 + }, + { + "budget": 0.02, + "threshold": -0.5456489324569702, + "operator": "gt", + "actual_fraction": 0.019765110283586365, + "metric": 0.8037811515325122, + "rescued": 51, + "harmed": 12, + "missed_rescue": 1036, + "avoided_harm": 371, + "selected_both_correct": 63, + "selected_both_wrong": 12 + }, + { + "budget": 0.03, + "threshold": -0.5652987360954285, + "operator": "ge", + "actual_fraction": 0.029934116299054712, + "metric": 0.8066456602692639, + "rescued": 80, + "harmed": 21, + "missed_rescue": 1007, + "avoided_harm": 362, + "selected_both_correct": 86, + "selected_both_wrong": 22 + }, + { + "budget": 0.04, + "threshold": -0.5851011276245117, + "operator": "ge", + "actual_fraction": 0.039959896877685475, + "metric": 0.8083643655113149, + "rescued": 101, + "harmed": 30, + "missed_rescue": 986, + "avoided_harm": 353, + "selected_both_correct": 118, + "selected_both_wrong": 30 + }, + { + "budget": 0.05, + "threshold": -0.6055658459663391, + "operator": "ge", + "actual_fraction": 0.04998567745631624, + "metric": 0.809653394442853, + "rescued": 121, + "harmed": 41, + "missed_rescue": 966, + "avoided_harm": 342, + "selected_both_correct": 151, + "selected_both_wrong": 36 + }, + { + "budget": 0.06, + "threshold": -0.6293190717697144, + "operator": "ge", + "actual_fraction": 0.059868232598109423, + "metric": 0.8125179031796047, + "rescued": 145, + "harmed": 45, + "missed_rescue": 942, + "avoided_harm": 338, + "selected_both_correct": 183, + "selected_both_wrong": 45 + }, + { + "budget": 0.07, + "threshold": -0.6477982401847839, + "operator": "ge", + "actual_fraction": 0.06989401317674018, + "metric": 0.8168146662847322, + "rescued": 177, + "harmed": 47, + "missed_rescue": 910, + "avoided_harm": 336, + "selected_both_correct": 212, + "selected_both_wrong": 52 + }, + { + "budget": 0.08, + "threshold": -0.6693127155303955, + "operator": "ge", + "actual_fraction": 0.07991979375537095, + "metric": 0.8188198224004584, + "rescued": 201, + "harmed": 57, + "missed_rescue": 886, + "avoided_harm": 326, + "selected_both_correct": 239, + "selected_both_wrong": 61 + }, + { + "budget": 0.09, + "threshold": -0.6888871788978577, + "operator": "gt", + "actual_fraction": 0.08951589802348897, + "metric": 0.8205385276425093, + "rescued": 222, + "harmed": 66, + "missed_rescue": 865, + "avoided_harm": 317, + "selected_both_correct": 267, + "selected_both_wrong": 70 + }, + { + "budget": 0.1, + "threshold": -0.7061904072761536, + "operator": "gt", + "actual_fraction": 0.09925522772844457, + "metric": 0.8222572328845603, + "rescued": 240, + "harmed": 72, + "missed_rescue": 847, + "avoided_harm": 311, + "selected_both_correct": 305, + "selected_both_wrong": 76 + }, + { + "budget": 0.11, + "threshold": -0.7233095169067383, + "operator": "ge", + "actual_fraction": 0.10999713549126325, + "metric": 0.8254081924949871, + "rescued": 270, + "harmed": 80, + "missed_rescue": 817, + "avoided_harm": 303, + "selected_both_correct": 339, + "selected_both_wrong": 79 + }, + { + "budget": 0.12, + "threshold": -0.7424216866493225, + "operator": "ge", + "actual_fraction": 0.11987969063305644, + "metric": 0.8282727012317388, + "rescued": 296, + "harmed": 86, + "missed_rescue": 791, + "avoided_harm": 297, + "selected_both_correct": 369, + "selected_both_wrong": 86 + }, + { + "budget": 0.13, + "threshold": -0.7627776861190796, + "operator": "gt", + "actual_fraction": 0.1297622457748496, + "metric": 0.8308507590948152, + "rescued": 320, + "harmed": 92, + "missed_rescue": 767, + "avoided_harm": 291, + "selected_both_correct": 404, + "selected_both_wrong": 90 + }, + { + "budget": 0.14, + "threshold": -0.7786493301391602, + "operator": "gt", + "actual_fraction": 0.1396448009166428, + "metric": 0.8337152678315669, + "rescued": 346, + "harmed": 98, + "missed_rescue": 741, + "avoided_harm": 285, + "selected_both_correct": 436, + "selected_both_wrong": 95 + }, + { + "budget": 0.15, + "threshold": -0.7950222492218018, + "operator": "ge", + "actual_fraction": 0.14995703236894872, + "metric": 0.8362933256946433, + "rescued": 367, + "harmed": 101, + "missed_rescue": 720, + "avoided_harm": 282, + "selected_both_correct": 474, + "selected_both_wrong": 105 + }, + { + "budget": 0.16, + "threshold": -0.8140527606010437, + "operator": "ge", + "actual_fraction": 0.1599828129475795, + "metric": 0.8394442853050702, + "rescued": 392, + "harmed": 104, + "missed_rescue": 695, + "avoided_harm": 279, + "selected_both_correct": 513, + "selected_both_wrong": 108 + }, + { + "budget": 0.17, + "threshold": -0.8327813148498535, + "operator": "ge", + "actual_fraction": 0.16986536808937266, + "metric": 0.840876539673446, + "rescued": 409, + "harmed": 111, + "missed_rescue": 678, + "avoided_harm": 272, + "selected_both_correct": 552, + "selected_both_wrong": 114 + }, + { + "budget": 0.18, + "threshold": -0.8464508652687073, + "operator": "gt", + "actual_fraction": 0.17960469779432828, + "metric": 0.8428816957891722, + "rescued": 426, + "harmed": 114, + "missed_rescue": 661, + "avoided_harm": 269, + "selected_both_correct": 596, + "selected_both_wrong": 118 + }, + { + "budget": 0.19, + "threshold": -0.858244001865387, + "operator": "ge", + "actual_fraction": 0.1899169292466342, + "metric": 0.8451733027785735, + "rescued": 449, + "harmed": 121, + "missed_rescue": 638, + "avoided_harm": 262, + "selected_both_correct": 631, + "selected_both_wrong": 125 + }, + { + "budget": 0.2, + "threshold": -0.8701565861701965, + "operator": "ge", + "actual_fraction": 0.19994270982526496, + "metric": 0.8467487825837868, + "rescued": 467, + "harmed": 128, + "missed_rescue": 620, + "avoided_harm": 255, + "selected_both_correct": 672, + "selected_both_wrong": 129 + }, + { + "budget": 0.21, + "threshold": -0.8840392827987671, + "operator": "gt", + "actual_fraction": 0.2093955886565454, + "metric": 0.8481810369521627, + "rescued": 483, + "harmed": 134, + "missed_rescue": 604, + "avoided_harm": 249, + "selected_both_correct": 712, + "selected_both_wrong": 133 + }, + { + "budget": 0.22, + "threshold": -0.8933094143867493, + "operator": "ge", + "actual_fraction": 0.2199942709825265, + "metric": 0.8497565167573761, + "rescued": 500, + "harmed": 140, + "missed_rescue": 587, + "avoided_harm": 243, + "selected_both_correct": 759, + "selected_both_wrong": 137 + }, + { + "budget": 0.23, + "threshold": -0.9046505093574524, + "operator": "gt", + "actual_fraction": 0.2297336006874821, + "metric": 0.851475221999427, + "rescued": 516, + "harmed": 144, + "missed_rescue": 571, + "avoided_harm": 239, + "selected_both_correct": 797, + "selected_both_wrong": 147 + }, + { + "budget": 0.24, + "threshold": -0.9136765599250793, + "operator": "ge", + "actual_fraction": 0.23990260670295044, + "metric": 0.8543397307361787, + "rescued": 540, + "harmed": 148, + "missed_rescue": 547, + "avoided_harm": 235, + "selected_both_correct": 834, + "selected_both_wrong": 153 + }, + { + "budget": 0.25, + "threshold": -0.9230391979217529, + "operator": "gt", + "actual_fraction": 0.24906903466055572, + "metric": 0.8562016614150673, + "rescued": 555, + "harmed": 150, + "missed_rescue": 532, + "avoided_harm": 233, + "selected_both_correct": 874, + "selected_both_wrong": 160 + }, + { + "budget": 0.26, + "threshold": -0.9314625263214111, + "operator": "gt", + "actual_fraction": 0.2595244915496992, + "metric": 0.8589229447149814, + "rescued": 581, + "harmed": 157, + "missed_rescue": 506, + "avoided_harm": 226, + "selected_both_correct": 913, + "selected_both_wrong": 161 + }, + { + "budget": 0.27, + "threshold": -0.9376689791679382, + "operator": "ge", + "actual_fraction": 0.26997994843884277, + "metric": 0.8607848753938699, + "rescued": 601, + "harmed": 164, + "missed_rescue": 486, + "avoided_harm": 219, + "selected_both_correct": 957, + "selected_both_wrong": 163 + }, + { + "budget": 0.28, + "threshold": -0.9449946880340576, + "operator": "gt", + "actual_fraction": 0.2794328272701232, + "metric": 0.8625035806359209, + "rescued": 616, + "harmed": 167, + "missed_rescue": 471, + "avoided_harm": 216, + "selected_both_correct": 1000, + "selected_both_wrong": 168 + }, + { + "budget": 0.29, + "threshold": -0.9511421918869019, + "operator": "gt", + "actual_fraction": 0.2896018332855915, + "metric": 0.8642222858779719, + "rescued": 633, + "harmed": 172, + "missed_rescue": 454, + "avoided_harm": 211, + "selected_both_correct": 1041, + "selected_both_wrong": 176 + }, + { + "budget": 0.3, + "threshold": -0.9559813141822815, + "operator": "gt", + "actual_fraction": 0.29977083930105985, + "metric": 0.8660842165568605, + "rescued": 652, + "harmed": 178, + "missed_rescue": 435, + "avoided_harm": 205, + "selected_both_correct": 1082, + "selected_both_wrong": 181 + }, + { + "budget": 0.31, + "threshold": -0.9597620368003845, + "operator": "ge", + "actual_fraction": 0.30993984531652824, + "metric": 0.8672300200515611, + "rescued": 664, + "harmed": 182, + "missed_rescue": 423, + "avoided_harm": 201, + "selected_both_correct": 1131, + "selected_both_wrong": 187 + }, + { + "budget": 0.32, + "threshold": -0.9643214344978333, + "operator": "gt", + "actual_fraction": 0.3196791750214838, + "metric": 0.8692351761672873, + "rescued": 682, + "harmed": 186, + "missed_rescue": 405, + "avoided_harm": 197, + "selected_both_correct": 1172, + "selected_both_wrong": 192 + }, + { + "budget": 0.33, + "threshold": -0.9686195850372314, + "operator": "gt", + "actual_fraction": 0.3297049556001146, + "metric": 0.8695216270409625, + "rescued": 693, + "harmed": 195, + "missed_rescue": 394, + "avoided_harm": 188, + "selected_both_correct": 1217, + "selected_both_wrong": 197 + }, + { + "budget": 0.34, + "threshold": -0.9719924926757812, + "operator": "gt", + "actual_fraction": 0.339157834431395, + "metric": 0.8712403322830135, + "rescued": 709, + "harmed": 199, + "missed_rescue": 378, + "avoided_harm": 184, + "selected_both_correct": 1257, + "selected_both_wrong": 203 + }, + { + "budget": 0.35, + "threshold": -0.974821150302887, + "operator": "ge", + "actual_fraction": 0.3498997421942137, + "metric": 0.872099684904039, + "rescued": 722, + "harmed": 206, + "missed_rescue": 365, + "avoided_harm": 177, + "selected_both_correct": 1305, + "selected_both_wrong": 210 + }, + { + "budget": 0.36, + "threshold": -0.9777138829231262, + "operator": "gt", + "actual_fraction": 0.3587797192781438, + "metric": 0.8739616155829275, + "rescued": 739, + "harmed": 210, + "missed_rescue": 348, + "avoided_harm": 173, + "selected_both_correct": 1340, + "selected_both_wrong": 216 + }, + { + "budget": 0.37, + "threshold": -0.9802808165550232, + "operator": "gt", + "actual_fraction": 0.3685190489830994, + "metric": 0.8746777427671154, + "rescued": 749, + "harmed": 215, + "missed_rescue": 338, + "avoided_harm": 168, + "selected_both_correct": 1389, + "selected_both_wrong": 220 + }, + { + "budget": 0.38, + "threshold": -0.982557475566864, + "operator": "gt", + "actual_fraction": 0.3795474076195932, + "metric": 0.8771125751933544, + "rescued": 769, + "harmed": 218, + "missed_rescue": 318, + "avoided_harm": 165, + "selected_both_correct": 1439, + "selected_both_wrong": 224 + }, + { + "budget": 0.39, + "threshold": -0.9843363761901855, + "operator": "gt", + "actual_fraction": 0.3892867373245488, + "metric": 0.8781151532512174, + "rescued": 780, + "harmed": 222, + "missed_rescue": 307, + "avoided_harm": 161, + "selected_both_correct": 1488, + "selected_both_wrong": 228 + }, + { + "budget": 0.4, + "threshold": -0.9863631129264832, + "operator": "gt", + "actual_fraction": 0.3993125179031796, + "metric": 0.8795474076195933, + "rescued": 796, + "harmed": 228, + "missed_rescue": 291, + "avoided_harm": 155, + "selected_both_correct": 1535, + "selected_both_wrong": 229 + }, + { + "budget": 0.41, + "threshold": -0.9879462122917175, + "operator": "gt", + "actual_fraction": 0.40962474935548554, + "metric": 0.8811228874248066, + "rescued": 812, + "harmed": 233, + "missed_rescue": 275, + "avoided_harm": 150, + "selected_both_correct": 1583, + "selected_both_wrong": 232 + }, + { + "budget": 0.42, + "threshold": -0.9890130758285522, + "operator": "gt", + "actual_fraction": 0.41850472643941566, + "metric": 0.8825551417931825, + "rescued": 826, + "harmed": 237, + "missed_rescue": 261, + "avoided_harm": 146, + "selected_both_correct": 1624, + "selected_both_wrong": 235 + }, + { + "budget": 0.43, + "threshold": -0.9899863600730896, + "operator": "gt", + "actual_fraction": 0.4270982526496706, + "metric": 0.8832712689773704, + "rescued": 834, + "harmed": 240, + "missed_rescue": 253, + "avoided_harm": 143, + "selected_both_correct": 1671, + "selected_both_wrong": 237 + }, + { + "budget": 0.44, + "threshold": -0.9911527037620544, + "operator": "gt", + "actual_fraction": 0.4395588656545402, + "metric": 0.8864222285877972, + "rescued": 859, + "harmed": 243, + "missed_rescue": 228, + "avoided_harm": 140, + "selected_both_correct": 1727, + "selected_both_wrong": 240 + }, + { + "budget": 0.45, + "threshold": -0.9919379353523254, + "operator": "gt", + "actual_fraction": 0.4497278716700086, + "metric": 0.8865654540246348, + "rescued": 866, + "harmed": 249, + "missed_rescue": 221, + "avoided_harm": 134, + "selected_both_correct": 1781, + "selected_both_wrong": 244 + }, + { + "budget": 0.46, + "threshold": -0.9926542043685913, + "operator": "gt", + "actual_fraction": 0.4586078487539387, + "metric": 0.8877112575193354, + "rescued": 876, + "harmed": 251, + "missed_rescue": 211, + "avoided_harm": 132, + "selected_both_correct": 1828, + "selected_both_wrong": 247 + }, + { + "budget": 0.47, + "threshold": -0.9933071732521057, + "operator": "gt", + "actual_fraction": 0.46863362933256947, + "metric": 0.888570610140361, + "rescued": 886, + "harmed": 255, + "missed_rescue": 201, + "avoided_harm": 128, + "selected_both_correct": 1882, + "selected_both_wrong": 249 + }, + { + "budget": 0.48, + "threshold": -0.9939024448394775, + "operator": "gt", + "actual_fraction": 0.4798052134059009, + "metric": 0.8882841592666858, + "rescued": 893, + "harmed": 264, + "missed_rescue": 194, + "avoided_harm": 119, + "selected_both_correct": 1942, + "selected_both_wrong": 251 + }, + { + "budget": 0.49, + "threshold": -0.9944451451301575, + "operator": "gt", + "actual_fraction": 0.4891148668003437, + "metric": 0.8897164136350616, + "rescued": 908, + "harmed": 269, + "missed_rescue": 179, + "avoided_harm": 114, + "selected_both_correct": 1986, + "selected_both_wrong": 252 + }, + { + "budget": 0.5, + "threshold": -0.9949396848678589, + "operator": "gt", + "actual_fraction": 0.4992838728158121, + "metric": 0.890289315382412, + "rescued": 915, + "harmed": 272, + "missed_rescue": 172, + "avoided_harm": 111, + "selected_both_correct": 2043, + "selected_both_wrong": 256 + }, + { + "budget": 0.51, + "threshold": -0.9952448010444641, + "operator": "gt", + "actual_fraction": 0.5065883700945288, + "metric": 0.890289315382412, + "rescued": 918, + "harmed": 275, + "missed_rescue": 169, + "avoided_harm": 108, + "selected_both_correct": 2085, + "selected_both_wrong": 259 + }, + { + "budget": 0.52, + "threshold": -0.9956685304641724, + "operator": "gt", + "actual_fraction": 0.5179031796046978, + "metric": 0.890575766256087, + "rescued": 923, + "harmed": 278, + "missed_rescue": 164, + "avoided_harm": 105, + "selected_both_correct": 2153, + "selected_both_wrong": 262 + }, + { + "budget": 0.53, + "threshold": -0.9961755275726318, + "operator": "gt", + "actual_fraction": 0.528788312804354, + "metric": 0.8911486680034374, + "rescued": 931, + "harmed": 282, + "missed_rescue": 156, + "avoided_harm": 101, + "selected_both_correct": 2215, + "selected_both_wrong": 264 + }, + { + "budget": 0.54, + "threshold": -0.9965165853500366, + "operator": "gt", + "actual_fraction": 0.5380979661987969, + "metric": 0.8921512460613005, + "rescued": 939, + "harmed": 283, + "missed_rescue": 148, + "avoided_harm": 100, + "selected_both_correct": 2270, + "selected_both_wrong": 265 + }, + { + "budget": 0.55, + "threshold": -0.9968273043632507, + "operator": "gt", + "actual_fraction": 0.5489830993984531, + "metric": 0.8920080206244629, + "rescued": 943, + "harmed": 288, + "missed_rescue": 144, + "avoided_harm": 95, + "selected_both_correct": 2337, + "selected_both_wrong": 265 + }, + { + "budget": 0.56, + "threshold": -0.9971103668212891, + "operator": "gt", + "actual_fraction": 0.5592953308507591, + "metric": 0.8922944714981381, + "rescued": 950, + "harmed": 293, + "missed_rescue": 137, + "avoided_harm": 90, + "selected_both_correct": 2393, + "selected_both_wrong": 269 + }, + { + "budget": 0.57, + "threshold": -0.997285008430481, + "operator": "gt", + "actual_fraction": 0.5690346605557147, + "metric": 0.8925809223718132, + "rescued": 957, + "harmed": 298, + "missed_rescue": 130, + "avoided_harm": 85, + "selected_both_correct": 2445, + "selected_both_wrong": 273 + }, + { + "budget": 0.58, + "threshold": -0.9974491000175476, + "operator": "gt", + "actual_fraction": 0.5783443139501575, + "metric": 0.8928673732454884, + "rescued": 961, + "harmed": 300, + "missed_rescue": 126, + "avoided_harm": 83, + "selected_both_correct": 2499, + "selected_both_wrong": 278 + }, + { + "budget": 0.59, + "threshold": -0.9976769089698792, + "operator": "gt", + "actual_fraction": 0.5893726725866514, + "metric": 0.8927241478086508, + "rescued": 965, + "harmed": 305, + "missed_rescue": 122, + "avoided_harm": 78, + "selected_both_correct": 2565, + "selected_both_wrong": 280 + }, + { + "budget": 0.6, + "threshold": -0.9978172779083252, + "operator": "gt", + "actual_fraction": 0.5976797479232312, + "metric": 0.8935835004296763, + "rescued": 972, + "harmed": 306, + "missed_rescue": 115, + "avoided_harm": 77, + "selected_both_correct": 2614, + "selected_both_wrong": 281 + }, + { + "budget": 0.61, + "threshold": -0.9980121850967407, + "operator": "gt", + "actual_fraction": 0.6084216556860499, + "metric": 0.8944428530507018, + "rescued": 980, + "harmed": 308, + "missed_rescue": 107, + "avoided_harm": 75, + "selected_both_correct": 2678, + "selected_both_wrong": 282 + }, + { + "budget": 0.62, + "threshold": -0.9981897473335266, + "operator": "gt", + "actual_fraction": 0.6187338871383558, + "metric": 0.8941564021770266, + "rescued": 982, + "harmed": 312, + "missed_rescue": 105, + "avoided_harm": 71, + "selected_both_correct": 2743, + "selected_both_wrong": 283 + }, + { + "budget": 0.63, + "threshold": -0.9982993006706238, + "operator": "gt", + "actual_fraction": 0.6280435405327986, + "metric": 0.8954454311085649, + "rescued": 992, + "harmed": 313, + "missed_rescue": 95, + "avoided_harm": 70, + "selected_both_correct": 2796, + "selected_both_wrong": 284 + }, + { + "budget": 0.64, + "threshold": -0.9984021782875061, + "operator": "gt", + "actual_fraction": 0.6360641649957033, + "metric": 0.8958751074190776, + "rescued": 997, + "harmed": 315, + "missed_rescue": 90, + "avoided_harm": 68, + "selected_both_correct": 2843, + "selected_both_wrong": 286 + }, + { + "budget": 0.65, + "threshold": -0.9985449314117432, + "operator": "gt", + "actual_fraction": 0.6465196218848468, + "metric": 0.8961615582927528, + "rescued": 1000, + "harmed": 316, + "missed_rescue": 87, + "avoided_harm": 67, + "selected_both_correct": 2912, + "selected_both_wrong": 286 + }, + { + "budget": 0.66, + "threshold": -0.9986749291419983, + "operator": "gt", + "actual_fraction": 0.6586937840160413, + "metric": 0.8971641363506159, + "rescued": 1011, + "harmed": 320, + "missed_rescue": 76, + "avoided_harm": 63, + "selected_both_correct": 2982, + "selected_both_wrong": 286 + }, + { + "budget": 0.67, + "threshold": -0.9987550973892212, + "operator": "gt", + "actual_fraction": 0.6672873102262962, + "metric": 0.8973073617874534, + "rescued": 1015, + "harmed": 323, + "missed_rescue": 72, + "avoided_harm": 60, + "selected_both_correct": 3034, + "selected_both_wrong": 287 + }, + { + "budget": 0.68, + "threshold": -0.9988664388656616, + "operator": "gt", + "actual_fraction": 0.6793182469206531, + "metric": 0.8977370380979662, + "rescued": 1023, + "harmed": 328, + "missed_rescue": 64, + "avoided_harm": 55, + "selected_both_correct": 3102, + "selected_both_wrong": 290 + }, + { + "budget": 0.69, + "threshold": -0.9989677667617798, + "operator": "gt", + "actual_fraction": 0.6894872529361215, + "metric": 0.8984531652821541, + "rescued": 1028, + "harmed": 328, + "missed_rescue": 59, + "avoided_harm": 55, + "selected_both_correct": 3166, + "selected_both_wrong": 292 + }, + { + "budget": 0.7, + "threshold": -0.9990302324295044, + "operator": "gt", + "actual_fraction": 0.6969349756516757, + "metric": 0.8975938126611286, + "rescued": 1028, + "harmed": 334, + "missed_rescue": 59, + "avoided_harm": 49, + "selected_both_correct": 3212, + "selected_both_wrong": 292 + }, + { + "budget": 0.71, + "threshold": -0.9991169571876526, + "operator": "gt", + "actual_fraction": 0.7098252649670581, + "metric": 0.8977370380979662, + "rescued": 1031, + "harmed": 336, + "missed_rescue": 56, + "avoided_harm": 47, + "selected_both_correct": 3292, + "selected_both_wrong": 297 + }, + { + "budget": 0.72, + "threshold": -0.9991704225540161, + "operator": "gt", + "actual_fraction": 0.7189916929246635, + "metric": 0.8978802635348038, + "rescued": 1034, + "harmed": 338, + "missed_rescue": 53, + "avoided_harm": 45, + "selected_both_correct": 3351, + "selected_both_wrong": 297 + }, + { + "budget": 0.73, + "threshold": -0.9992445707321167, + "operator": "gt", + "actual_fraction": 0.7295903752506445, + "metric": 0.898166714408479, + "rescued": 1038, + "harmed": 340, + "missed_rescue": 49, + "avoided_harm": 43, + "selected_both_correct": 3419, + "selected_both_wrong": 297 + }, + { + "budget": 0.74, + "threshold": -0.9992903470993042, + "operator": "gt", + "actual_fraction": 0.7384703523345746, + "metric": 0.8985963907189917, + "rescued": 1044, + "harmed": 343, + "missed_rescue": 43, + "avoided_harm": 40, + "selected_both_correct": 3471, + "selected_both_wrong": 298 + }, + { + "budget": 0.75, + "threshold": -0.9993333220481873, + "operator": "gt", + "actual_fraction": 0.7484961329132054, + "metric": 0.8988828415926668, + "rescued": 1049, + "harmed": 346, + "missed_rescue": 38, + "avoided_harm": 37, + "selected_both_correct": 3533, + "selected_both_wrong": 298 + }, + { + "budget": 0.76, + "threshold": -0.999373733997345, + "operator": "gt", + "actual_fraction": 0.7553709538814093, + "metric": 0.8987396161558293, + "rescued": 1050, + "harmed": 348, + "missed_rescue": 37, + "avoided_harm": 35, + "selected_both_correct": 3578, + "selected_both_wrong": 298 + }, + { + "budget": 0.77, + "threshold": -0.9994296431541443, + "operator": "gt", + "actual_fraction": 0.7694070466914924, + "metric": 0.8994557433400172, + "rescued": 1056, + "harmed": 349, + "missed_rescue": 31, + "avoided_harm": 34, + "selected_both_correct": 3667, + "selected_both_wrong": 300 + }, + { + "budget": 0.78, + "threshold": -0.999464213848114, + "operator": "gt", + "actual_fraction": 0.7782870237754225, + "metric": 0.8995989687768547, + "rescued": 1058, + "harmed": 350, + "missed_rescue": 29, + "avoided_harm": 33, + "selected_both_correct": 3723, + "selected_both_wrong": 303 + }, + { + "budget": 0.79, + "threshold": -0.9994966983795166, + "operator": "gt", + "actual_fraction": 0.7885992552277284, + "metric": 0.8998854196505299, + "rescued": 1060, + "harmed": 350, + "missed_rescue": 27, + "avoided_harm": 33, + "selected_both_correct": 3792, + "selected_both_wrong": 304 + }, + { + "budget": 0.8, + "threshold": -0.9995417594909668, + "operator": "gt", + "actual_fraction": 0.7977656831853337, + "metric": 0.8997421942136924, + "rescued": 1061, + "harmed": 352, + "missed_rescue": 26, + "avoided_harm": 31, + "selected_both_correct": 3852, + "selected_both_wrong": 305 + }, + { + "budget": 0.81, + "threshold": -0.9995827078819275, + "operator": "gt", + "actual_fraction": 0.8090804926955028, + "metric": 0.9001718705242051, + "rescued": 1064, + "harmed": 352, + "missed_rescue": 23, + "avoided_harm": 31, + "selected_both_correct": 3928, + "selected_both_wrong": 305 + }, + { + "budget": 0.82, + "threshold": -0.9996140599250793, + "operator": "gt", + "actual_fraction": 0.8198224004583214, + "metric": 0.9003150959610426, + "rescued": 1066, + "harmed": 353, + "missed_rescue": 21, + "avoided_harm": 30, + "selected_both_correct": 3998, + "selected_both_wrong": 307 + }, + { + "budget": 0.83, + "threshold": -0.9996429681777954, + "operator": "gt", + "actual_fraction": 0.8281294757949011, + "metric": 0.9001718705242051, + "rescued": 1066, + "harmed": 354, + "missed_rescue": 21, + "avoided_harm": 29, + "selected_both_correct": 4054, + "selected_both_wrong": 308 + }, + { + "budget": 0.84, + "threshold": -0.9996750354766846, + "operator": "gt", + "actual_fraction": 0.8395875107419077, + "metric": 0.9003150959610426, + "rescued": 1070, + "harmed": 357, + "missed_rescue": 17, + "avoided_harm": 26, + "selected_both_correct": 4127, + "selected_both_wrong": 308 + }, + { + "budget": 0.85, + "threshold": -0.9997040629386902, + "operator": "gt", + "actual_fraction": 0.8458894299627614, + "metric": 0.9003150959610426, + "rescued": 1071, + "harmed": 358, + "missed_rescue": 16, + "avoided_harm": 25, + "selected_both_correct": 4167, + "selected_both_wrong": 310 + }, + { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8587797192781438, + "metric": 0.9004583213978803, + "rescued": 1073, + "harmed": 359, + "missed_rescue": 14, + "avoided_harm": 24, + "selected_both_correct": 4254, + "selected_both_wrong": 310 + }, + { + "budget": 0.87, + "threshold": -0.9997695088386536, + "operator": "gt", + "actual_fraction": 0.8693784016041249, + "metric": 0.9000286450873675, + "rescued": 1073, + "harmed": 362, + "missed_rescue": 14, + "avoided_harm": 21, + "selected_both_correct": 4323, + "selected_both_wrong": 312 + }, + { + "budget": 0.88, + "threshold": -0.999783456325531, + "operator": "gt", + "actual_fraction": 0.8749641936407906, + "metric": 0.8998854196505299, + "rescued": 1073, + "harmed": 363, + "missed_rescue": 14, + "avoided_harm": 20, + "selected_both_correct": 4361, + "selected_both_wrong": 312 + }, + { + "budget": 0.89, + "threshold": -0.9998205304145813, + "operator": "gt", + "actual_fraction": 0.8894299627613864, + "metric": 0.8994557433400172, + "rescued": 1074, + "harmed": 367, + "missed_rescue": 13, + "avoided_harm": 16, + "selected_both_correct": 4456, + "selected_both_wrong": 313 + }, + { + "budget": 0.9, + "threshold": -0.999841570854187, + "operator": "gt", + "actual_fraction": 0.8988828415926668, + "metric": 0.8993125179031796, + "rescued": 1075, + "harmed": 369, + "missed_rescue": 12, + "avoided_harm": 14, + "selected_both_correct": 4517, + "selected_both_wrong": 315 + }, + { + "budget": 0.91, + "threshold": -0.9998601675033569, + "operator": "gt", + "actual_fraction": 0.9093382984818104, + "metric": 0.8994557433400172, + "rescued": 1077, + "harmed": 370, + "missed_rescue": 10, + "avoided_harm": 13, + "selected_both_correct": 4586, + "selected_both_wrong": 316 + }, + { + "budget": 0.92, + "threshold": -0.9998766183853149, + "operator": "gt", + "actual_fraction": 0.9185047264394156, + "metric": 0.8993125179031796, + "rescued": 1077, + "harmed": 371, + "missed_rescue": 10, + "avoided_harm": 12, + "selected_both_correct": 4646, + "selected_both_wrong": 319 + }, + { + "budget": 0.93, + "threshold": -0.9998977184295654, + "operator": "gt", + "actual_fraction": 0.9276711543970209, + "metric": 0.899169292466342, + "rescued": 1078, + "harmed": 373, + "missed_rescue": 9, + "avoided_harm": 10, + "selected_both_correct": 4707, + "selected_both_wrong": 319 + }, + { + "budget": 0.94, + "threshold": -0.9999151229858398, + "operator": "gt", + "actual_fraction": 0.9394156402177026, + "metric": 0.8993125179031796, + "rescued": 1081, + "harmed": 375, + "missed_rescue": 6, + "avoided_harm": 8, + "selected_both_correct": 4784, + "selected_both_wrong": 319 + }, + { + "budget": 0.95, + "threshold": -0.999931812286377, + "operator": "gt", + "actual_fraction": 0.9497278716700086, + "metric": 0.8993125179031796, + "rescued": 1082, + "harmed": 376, + "missed_rescue": 5, + "avoided_harm": 7, + "selected_both_correct": 4854, + "selected_both_wrong": 319 + }, + { + "budget": 0.96, + "threshold": -0.9999452829360962, + "operator": "gt", + "actual_fraction": 0.9590375250644515, + "metric": 0.8995989687768547, + "rescued": 1085, + "harmed": 377, + "missed_rescue": 2, + "avoided_harm": 6, + "selected_both_correct": 4914, + "selected_both_wrong": 320 + }, + { + "budget": 0.97, + "threshold": -0.9999599456787109, + "operator": "gt", + "actual_fraction": 0.969492981953595, + "metric": 0.8994557433400172, + "rescued": 1085, + "harmed": 378, + "missed_rescue": 2, + "avoided_harm": 5, + "selected_both_correct": 4986, + "selected_both_wrong": 320 + }, + { + "budget": 0.98, + "threshold": -0.9999687671661377, + "operator": "gt", + "actual_fraction": 0.9783729590375251, + "metric": 0.899169292466342, + "rescued": 1085, + "harmed": 380, + "missed_rescue": 2, + "avoided_harm": 3, + "selected_both_correct": 5046, + "selected_both_wrong": 320 + }, + { + "budget": 0.99, + "threshold": -0.9999842643737793, + "operator": "gt", + "actual_fraction": 0.9898309939845317, + "metric": 0.899169292466342, + "rescued": 1087, + "harmed": 382, + "missed_rescue": 0, + "avoided_harm": 1, + "selected_both_correct": 5122, + "selected_both_wrong": 320 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8990260670295045, + "rescued": 1087, + "harmed": 383, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 5190, + "selected_both_wrong": 322 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.8000287005811868, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2225, + "avoided_harm": 729, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5245773792266846, + "operator": "gt", + "actual_fraction": 0.011623735380641457, + "metric": 0.8034727703235991, + "rescued": 65, + "harmed": 17, + "missed_rescue": 2160, + "avoided_harm": 712, + "selected_both_correct": 65, + "selected_both_wrong": 15 + }, + { + "budget": 0.02, + "threshold": -0.5456489324569702, + "operator": "gt", + "actual_fraction": 0.021453684437109852, + "metric": 0.8064145798952429, + "rescued": 115, + "harmed": 26, + "missed_rescue": 2110, + "avoided_harm": 703, + "selected_both_correct": 126, + "selected_both_wrong": 32 + }, + { + "budget": 0.03, + "threshold": -0.5652987360954285, + "operator": "ge", + "actual_fraction": 0.02970510152830595, + "metric": 0.8094281409198536, + "rescued": 165, + "harmed": 34, + "missed_rescue": 2060, + "avoided_harm": 695, + "selected_both_correct": 174, + "selected_both_wrong": 41 + }, + { + "budget": 0.04, + "threshold": -0.5851011276245117, + "operator": "ge", + "actual_fraction": 0.03795651861950204, + "metric": 0.8113654301499605, + "rescued": 204, + "harmed": 46, + "missed_rescue": 2021, + "avoided_harm": 683, + "selected_both_correct": 229, + "selected_both_wrong": 50 + }, + { + "budget": 0.05, + "threshold": -0.6055658459663391, + "operator": "ge", + "actual_fraction": 0.047212456052235056, + "metric": 0.8140202339097367, + "rescued": 253, + "harmed": 58, + "missed_rescue": 1972, + "avoided_harm": 671, + "selected_both_correct": 289, + "selected_both_wrong": 58 + }, + { + "budget": 0.06, + "threshold": -0.6293190717697144, + "operator": "ge", + "actual_fraction": 0.057759919638372674, + "metric": 0.817464303652149, + "rescued": 312, + "harmed": 69, + "missed_rescue": 1913, + "avoided_harm": 660, + "selected_both_correct": 351, + "selected_both_wrong": 73 + }, + { + "budget": 0.07, + "threshold": -0.6477982401847839, + "operator": "ge", + "actual_fraction": 0.06601133672956877, + "metric": 0.8196885986941236, + "rescued": 356, + "harmed": 82, + "missed_rescue": 1869, + "avoided_harm": 647, + "selected_both_correct": 401, + "selected_both_wrong": 81 + }, + { + "budget": 0.08, + "threshold": -0.6693127155303955, + "operator": "ge", + "actual_fraction": 0.07483676544450026, + "metric": 0.8226304082657674, + "rescued": 406, + "harmed": 91, + "missed_rescue": 1819, + "avoided_harm": 638, + "selected_both_correct": 453, + "selected_both_wrong": 93 + }, + { + "budget": 0.09, + "threshold": -0.6888871788978577, + "operator": "gt", + "actual_fraction": 0.08488196885986941, + "metric": 0.8254287149314774, + "rescued": 455, + "harmed": 101, + "missed_rescue": 1770, + "avoided_harm": 628, + "selected_both_correct": 523, + "selected_both_wrong": 104 + }, + { + "budget": 0.1, + "threshold": -0.7061904072761536, + "operator": "gt", + "actual_fraction": 0.09356389466886704, + "metric": 0.8282987730501543, + "rescued": 503, + "harmed": 109, + "missed_rescue": 1722, + "avoided_harm": 620, + "selected_both_correct": 576, + "selected_both_wrong": 116 + }, + { + "budget": 0.11, + "threshold": -0.7233095169067383, + "operator": "ge", + "actual_fraction": 0.1026045777426993, + "metric": 0.8310970797158642, + "rescued": 551, + "harmed": 118, + "missed_rescue": 1674, + "avoided_harm": 611, + "selected_both_correct": 638, + "selected_both_wrong": 123 + }, + { + "budget": 0.12, + "threshold": -0.7424216866493225, + "operator": "ge", + "actual_fraction": 0.11322379278180383, + "metric": 0.8338953863815742, + "rescued": 603, + "harmed": 131, + "missed_rescue": 1622, + "avoided_harm": 598, + "selected_both_correct": 708, + "selected_both_wrong": 136 + }, + { + "budget": 0.13, + "threshold": -0.7627776861190796, + "operator": "gt", + "actual_fraction": 0.12255148166750376, + "metric": 0.8363349357824496, + "rescued": 646, + "harmed": 140, + "missed_rescue": 1579, + "avoided_harm": 589, + "selected_both_correct": 776, + "selected_both_wrong": 146 + }, + { + "budget": 0.14, + "threshold": -0.7786493301391602, + "operator": "gt", + "actual_fraction": 0.13137691038243524, + "metric": 0.8390614909951927, + "rescued": 693, + "harmed": 149, + "missed_rescue": 1532, + "avoided_harm": 580, + "selected_both_correct": 832, + "selected_both_wrong": 157 + }, + { + "budget": 0.15, + "threshold": -0.7950222492218018, + "operator": "ge", + "actual_fraction": 0.14056109636220132, + "metric": 0.8419315491138696, + "rescued": 743, + "harmed": 159, + "missed_rescue": 1482, + "avoided_harm": 570, + "selected_both_correct": 891, + "selected_both_wrong": 166 + }, + { + "budget": 0.16, + "threshold": -0.8140527606010437, + "operator": "ge", + "actual_fraction": 0.15096505704240512, + "metric": 0.8448016072325465, + "rescued": 793, + "harmed": 169, + "missed_rescue": 1432, + "avoided_harm": 560, + "selected_both_correct": 965, + "selected_both_wrong": 177 + }, + { + "budget": 0.17, + "threshold": -0.8327813148498535, + "operator": "ge", + "actual_fraction": 0.1613690177226089, + "metric": 0.8465236421037526, + "rescued": 831, + "harmed": 183, + "missed_rescue": 1394, + "avoided_harm": 546, + "selected_both_correct": 1041, + "selected_both_wrong": 194 + }, + { + "budget": 0.18, + "threshold": -0.8464508652687073, + "operator": "gt", + "actual_fraction": 0.1702661978905073, + "metric": 0.8483891798808926, + "rescued": 871, + "harmed": 197, + "missed_rescue": 1354, + "avoided_harm": 532, + "selected_both_correct": 1104, + "selected_both_wrong": 201 + }, + { + "budget": 0.19, + "threshold": -0.858244001865387, + "operator": "ge", + "actual_fraction": 0.17916337805840568, + "metric": 0.8509722321877018, + "rescued": 915, + "harmed": 205, + "missed_rescue": 1310, + "avoided_harm": 524, + "selected_both_correct": 1164, + "selected_both_wrong": 213 + }, + { + "budget": 0.2, + "threshold": -0.8701565861701965, + "operator": "ge", + "actual_fraction": 0.18798880677333715, + "metric": 0.8528377699648418, + "rescued": 954, + "harmed": 218, + "missed_rescue": 1271, + "avoided_harm": 511, + "selected_both_correct": 1230, + "selected_both_wrong": 218 + }, + { + "budget": 0.21, + "threshold": -0.8840392827987671, + "operator": "gt", + "actual_fraction": 0.19659898112936788, + "metric": 0.8549903135538495, + "rescued": 992, + "harmed": 226, + "missed_rescue": 1233, + "avoided_harm": 503, + "selected_both_correct": 1299, + "selected_both_wrong": 223 + }, + { + "budget": 0.22, + "threshold": -0.8933094143867493, + "operator": "ge", + "actual_fraction": 0.20642893018583627, + "metric": 0.8574298629547248, + "rescued": 1035, + "harmed": 235, + "missed_rescue": 1190, + "avoided_harm": 494, + "selected_both_correct": 1377, + "selected_both_wrong": 230 + }, + { + "budget": 0.23, + "threshold": -0.9046505093574524, + "operator": "gt", + "actual_fraction": 0.21711989667790774, + "metric": 0.8599411638085671, + "rescued": 1079, + "harmed": 244, + "missed_rescue": 1146, + "avoided_harm": 485, + "selected_both_correct": 1461, + "selected_both_wrong": 242 + }, + { + "budget": 0.24, + "threshold": -0.9136765599250793, + "operator": "ge", + "actual_fraction": 0.22766736026404535, + "metric": 0.8620937073975748, + "rescued": 1120, + "harmed": 255, + "missed_rescue": 1105, + "avoided_harm": 474, + "selected_both_correct": 1539, + "selected_both_wrong": 259 + }, + { + "budget": 0.25, + "threshold": -0.9230391979217529, + "operator": "gt", + "actual_fraction": 0.23785606658534836, + "metric": 0.8648202626103179, + "rescued": 1170, + "harmed": 267, + "missed_rescue": 1055, + "avoided_harm": 462, + "selected_both_correct": 1612, + "selected_both_wrong": 266 + }, + { + "budget": 0.26, + "threshold": -0.9314625263214111, + "operator": "gt", + "actual_fraction": 0.24804477290665136, + "metric": 0.8671163091052594, + "rescued": 1211, + "harmed": 276, + "missed_rescue": 1014, + "avoided_harm": 453, + "selected_both_correct": 1693, + "selected_both_wrong": 277 + }, + { + "budget": 0.27, + "threshold": -0.9376689791679382, + "operator": "ge", + "actual_fraction": 0.25751596469828514, + "metric": 0.8688383439764655, + "rescued": 1245, + "harmed": 286, + "missed_rescue": 980, + "avoided_harm": 443, + "selected_both_correct": 1771, + "selected_both_wrong": 287 + }, + { + "budget": 0.28, + "threshold": -0.9449946880340576, + "operator": "gt", + "actual_fraction": 0.26863743990815814, + "metric": 0.8704886273947048, + "rescued": 1278, + "harmed": 296, + "missed_rescue": 947, + "avoided_harm": 433, + "selected_both_correct": 1869, + "selected_both_wrong": 301 + }, + { + "budget": 0.29, + "threshold": -0.9511421918869019, + "operator": "gt", + "actual_fraction": 0.2793284064002296, + "metric": 0.8725694195307455, + "rescued": 1322, + "harmed": 311, + "missed_rescue": 903, + "avoided_harm": 418, + "selected_both_correct": 1951, + "selected_both_wrong": 309 + }, + { + "budget": 0.3, + "threshold": -0.9559813141822815, + "operator": "gt", + "actual_fraction": 0.2897323670804334, + "metric": 0.8739326971371171, + "rescued": 1350, + "harmed": 320, + "missed_rescue": 875, + "avoided_harm": 409, + "selected_both_correct": 2050, + "selected_both_wrong": 318 + }, + { + "budget": 0.31, + "threshold": -0.9597620368003845, + "operator": "ge", + "actual_fraction": 0.2999928248547033, + "metric": 0.8755829805553562, + "rescued": 1377, + "harmed": 324, + "missed_rescue": 848, + "avoided_harm": 405, + "selected_both_correct": 2158, + "selected_both_wrong": 322 + }, + { + "budget": 0.32, + "threshold": -0.9643214344978333, + "operator": "gt", + "actual_fraction": 0.3091770108344694, + "metric": 0.8770897610676616, + "rescued": 1408, + "harmed": 334, + "missed_rescue": 817, + "avoided_harm": 395, + "selected_both_correct": 2234, + "selected_both_wrong": 333 + }, + { + "budget": 0.33, + "threshold": -0.9686195850372314, + "operator": "gt", + "actual_fraction": 0.3218770180096147, + "metric": 0.878596541579967, + "rescued": 1442, + "harmed": 347, + "missed_rescue": 783, + "avoided_harm": 382, + "selected_both_correct": 2358, + "selected_both_wrong": 339 + }, + { + "budget": 0.34, + "threshold": -0.9719924926757812, + "operator": "gt", + "actual_fraction": 0.33127645834828157, + "metric": 0.8806055822630409, + "rescued": 1476, + "harmed": 353, + "missed_rescue": 749, + "avoided_harm": 376, + "selected_both_correct": 2445, + "selected_both_wrong": 343 + }, + { + "budget": 0.35, + "threshold": -0.974821150302887, + "operator": "ge", + "actual_fraction": 0.3431871995407907, + "metric": 0.8816101026045777, + "rescued": 1507, + "harmed": 370, + "missed_rescue": 718, + "avoided_harm": 359, + "selected_both_correct": 2556, + "selected_both_wrong": 350 + }, + { + "budget": 0.36, + "threshold": -0.9777138829231262, + "operator": "gt", + "actual_fraction": 0.35280189423835834, + "metric": 0.8823993685872139, + "rescued": 1526, + "harmed": 378, + "missed_rescue": 699, + "avoided_harm": 351, + "selected_both_correct": 2652, + "selected_both_wrong": 361 + }, + { + "budget": 0.37, + "threshold": -0.9802808165550232, + "operator": "gt", + "actual_fraction": 0.36421037526009903, + "metric": 0.8837626461935855, + "rescued": 1555, + "harmed": 388, + "missed_rescue": 670, + "avoided_harm": 341, + "selected_both_correct": 2768, + "selected_both_wrong": 365 + }, + { + "budget": 0.38, + "threshold": -0.982557475566864, + "operator": "gt", + "actual_fraction": 0.3761928679055751, + "metric": 0.8851259237999569, + "rescued": 1587, + "harmed": 401, + "missed_rescue": 638, + "avoided_harm": 328, + "selected_both_correct": 2882, + "selected_both_wrong": 373 + }, + { + "budget": 0.39, + "threshold": -0.9843363761901855, + "operator": "gt", + "actual_fraction": 0.38803185764511733, + "metric": 0.8864892014063285, + "rescued": 1614, + "harmed": 409, + "missed_rescue": 611, + "avoided_harm": 320, + "selected_both_correct": 3004, + "selected_both_wrong": 381 + }, + { + "budget": 0.4, + "threshold": -0.9863631129264832, + "operator": "gt", + "actual_fraction": 0.4008036162732295, + "metric": 0.8881394848245677, + "rescued": 1647, + "harmed": 419, + "missed_rescue": 578, + "avoided_harm": 310, + "selected_both_correct": 3133, + "selected_both_wrong": 387 + }, + { + "budget": 0.41, + "threshold": -0.9879462122917175, + "operator": "gt", + "actual_fraction": 0.41156633421826794, + "metric": 0.8890722537131377, + "rescued": 1670, + "harmed": 429, + "missed_rescue": 555, + "avoided_harm": 300, + "selected_both_correct": 3241, + "selected_both_wrong": 396 + }, + { + "budget": 0.42, + "threshold": -0.9890130758285522, + "operator": "gt", + "actual_fraction": 0.4204635143861663, + "metric": 0.8895745138839062, + "rescued": 1684, + "harmed": 436, + "missed_rescue": 541, + "avoided_harm": 293, + "selected_both_correct": 3336, + "selected_both_wrong": 404 + }, + { + "budget": 0.43, + "threshold": -0.9899863600730896, + "operator": "gt", + "actual_fraction": 0.4293606945540647, + "metric": 0.890579034225443, + "rescued": 1706, + "harmed": 444, + "missed_rescue": 519, + "avoided_harm": 285, + "selected_both_correct": 3428, + "selected_both_wrong": 406 + }, + { + "budget": 0.44, + "threshold": -0.9911527037620544, + "operator": "gt", + "actual_fraction": 0.44119968429360695, + "metric": 0.89251632345555, + "rescued": 1742, + "harmed": 453, + "missed_rescue": 483, + "avoided_harm": 276, + "selected_both_correct": 3542, + "selected_both_wrong": 412 + }, + { + "budget": 0.45, + "threshold": -0.9919379353523254, + "operator": "gt", + "actual_fraction": 0.4516753964267776, + "metric": 0.8933773408911531, + "rescued": 1765, + "harmed": 464, + "missed_rescue": 460, + "avoided_harm": 265, + "selected_both_correct": 3649, + "selected_both_wrong": 417 + }, + { + "budget": 0.46, + "threshold": -0.9926542043685913, + "operator": "gt", + "actual_fraction": 0.4611465882184114, + "metric": 0.8938796010619215, + "rescued": 1781, + "harmed": 473, + "missed_rescue": 444, + "avoided_harm": 256, + "selected_both_correct": 3750, + "selected_both_wrong": 423 + }, + { + "budget": 0.47, + "threshold": -0.9933071732521057, + "operator": "gt", + "actual_fraction": 0.4706895314630121, + "metric": 0.8947406184975246, + "rescued": 1797, + "harmed": 477, + "missed_rescue": 428, + "avoided_harm": 252, + "selected_both_correct": 3860, + "selected_both_wrong": 426 + }, + { + "budget": 0.48, + "threshold": -0.9939024448394775, + "operator": "gt", + "actual_fraction": 0.481021740690249, + "metric": 0.8955298844801607, + "rescued": 1817, + "harmed": 486, + "missed_rescue": 408, + "avoided_harm": 243, + "selected_both_correct": 3969, + "selected_both_wrong": 432 + }, + { + "budget": 0.49, + "threshold": -0.9944451451301575, + "operator": "gt", + "actual_fraction": 0.4912821984645189, + "metric": 0.8963909019157638, + "rescued": 1835, + "harmed": 492, + "missed_rescue": 390, + "avoided_harm": 237, + "selected_both_correct": 4079, + "selected_both_wrong": 441 + }, + { + "budget": 0.5, + "threshold": -0.9949396848678589, + "operator": "gt", + "actual_fraction": 0.4992466097438473, + "metric": 0.8968931620865322, + "rescued": 1850, + "harmed": 500, + "missed_rescue": 375, + "avoided_harm": 229, + "selected_both_correct": 4163, + "selected_both_wrong": 445 + }, + { + "budget": 0.51, + "threshold": -0.9952448010444641, + "operator": "gt", + "actual_fraction": 0.5059912463227381, + "metric": 0.897108416445433, + "rescued": 1858, + "harmed": 505, + "missed_rescue": 367, + "avoided_harm": 224, + "selected_both_correct": 4239, + "selected_both_wrong": 450 + }, + { + "budget": 0.52, + "threshold": -0.9956685304641724, + "operator": "gt", + "actual_fraction": 0.5140991605080003, + "metric": 0.8974671737102676, + "rescued": 1869, + "harmed": 511, + "missed_rescue": 356, + "avoided_harm": 218, + "selected_both_correct": 4331, + "selected_both_wrong": 454 + }, + { + "budget": 0.53, + "threshold": -0.9961755275726318, + "operator": "gt", + "actual_fraction": 0.529166965631054, + "metric": 0.8992609600344407, + "rescued": 1903, + "harmed": 520, + "missed_rescue": 322, + "avoided_harm": 209, + "selected_both_correct": 4495, + "selected_both_wrong": 457 + }, + { + "budget": 0.54, + "threshold": -0.9965165853500366, + "operator": "gt", + "actual_fraction": 0.5406471981057617, + "metric": 0.899834971658176, + "rescued": 1920, + "harmed": 529, + "missed_rescue": 305, + "avoided_harm": 200, + "selected_both_correct": 4619, + "selected_both_wrong": 467 + }, + { + "budget": 0.55, + "threshold": -0.9968273043632507, + "operator": "gt", + "actual_fraction": 0.5504771471622301, + "metric": 0.89997847456411, + "rescued": 1927, + "harmed": 534, + "missed_rescue": 298, + "avoided_harm": 195, + "selected_both_correct": 4743, + "selected_both_wrong": 468 + }, + { + "budget": 0.56, + "threshold": -0.9971103668212891, + "operator": "gt", + "actual_fraction": 0.5621726339958384, + "metric": 0.9006959890937791, + "rescued": 1946, + "harmed": 543, + "missed_rescue": 279, + "avoided_harm": 186, + "selected_both_correct": 4874, + "selected_both_wrong": 472 + }, + { + "budget": 0.57, + "threshold": -0.997285008430481, + "operator": "gt", + "actual_fraction": 0.5677692473272584, + "metric": 0.9011264978115807, + "rescued": 1956, + "harmed": 547, + "missed_rescue": 269, + "avoided_harm": 182, + "selected_both_correct": 4935, + "selected_both_wrong": 475 + }, + { + "budget": 0.58, + "threshold": -0.9974491000175476, + "operator": "gt", + "actual_fraction": 0.5759489129654876, + "metric": 0.9014135036234484, + "rescued": 1968, + "harmed": 555, + "missed_rescue": 257, + "avoided_harm": 174, + "selected_both_correct": 5025, + "selected_both_wrong": 479 + }, + { + "budget": 0.59, + "threshold": -0.9976769089698792, + "operator": "gt", + "actual_fraction": 0.5864963765516251, + "metric": 0.9019875152471838, + "rescued": 1985, + "harmed": 564, + "missed_rescue": 240, + "avoided_harm": 165, + "selected_both_correct": 5144, + "selected_both_wrong": 481 + }, + { + "budget": 0.6, + "threshold": -0.9978172779083252, + "operator": "gt", + "actual_fraction": 0.5942455334720528, + "metric": 0.9022745210590515, + "rescued": 1994, + "harmed": 569, + "missed_rescue": 231, + "avoided_harm": 160, + "selected_both_correct": 5235, + "selected_both_wrong": 484 + }, + { + "budget": 0.61, + "threshold": -0.9980121850967407, + "operator": "gt", + "actual_fraction": 0.6054387601348927, + "metric": 0.9023462725120184, + "rescued": 2002, + "harmed": 576, + "missed_rescue": 223, + "avoided_harm": 153, + "selected_both_correct": 5372, + "selected_both_wrong": 488 + }, + { + "budget": 0.62, + "threshold": -0.9981897473335266, + "operator": "gt", + "actual_fraction": 0.6165602353447658, + "metric": 0.9033507928535552, + "rescued": 2018, + "harmed": 578, + "missed_rescue": 207, + "avoided_harm": 151, + "selected_both_correct": 5504, + "selected_both_wrong": 493 + }, + { + "budget": 0.63, + "threshold": -0.9982993006706238, + "operator": "gt", + "actual_fraction": 0.6242376408122264, + "metric": 0.9037095501183899, + "rescued": 2027, + "harmed": 582, + "missed_rescue": 198, + "avoided_harm": 147, + "selected_both_correct": 5596, + "selected_both_wrong": 495 + }, + { + "budget": 0.64, + "threshold": -0.9984021782875061, + "operator": "gt", + "actual_fraction": 0.6322020520915549, + "metric": 0.9040683073832245, + "rescued": 2037, + "harmed": 587, + "missed_rescue": 188, + "avoided_harm": 142, + "selected_both_correct": 5691, + "selected_both_wrong": 496 + }, + { + "budget": 0.65, + "threshold": -0.9985449314117432, + "operator": "gt", + "actual_fraction": 0.6454043194374686, + "metric": 0.9046423190069599, + "rescued": 2051, + "harmed": 593, + "missed_rescue": 174, + "avoided_harm": 136, + "selected_both_correct": 5853, + "selected_both_wrong": 498 + }, + { + "budget": 0.66, + "threshold": -0.9986749291419983, + "operator": "gt", + "actual_fraction": 0.6566692975532754, + "metric": 0.9050010762717945, + "rescued": 2062, + "harmed": 599, + "missed_rescue": 163, + "avoided_harm": 130, + "selected_both_correct": 5989, + "selected_both_wrong": 502 + }, + { + "budget": 0.67, + "threshold": -0.9987550973892212, + "operator": "gt", + "actual_fraction": 0.6646337088326039, + "metric": 0.9047858219128937, + "rescued": 2066, + "harmed": 606, + "missed_rescue": 159, + "avoided_harm": 123, + "selected_both_correct": 6087, + "selected_both_wrong": 504 + }, + { + "budget": 0.68, + "threshold": -0.9988664388656616, + "operator": "gt", + "actual_fraction": 0.6776207218196169, + "metric": 0.9051445791777284, + "rescued": 2077, + "harmed": 612, + "missed_rescue": 148, + "avoided_harm": 117, + "selected_both_correct": 6248, + "selected_both_wrong": 507 + }, + { + "budget": 0.69, + "threshold": -0.9989677667617798, + "operator": "gt", + "actual_fraction": 0.690464231900696, + "metric": 0.9052163306306953, + "rescued": 2084, + "harmed": 618, + "missed_rescue": 141, + "avoided_harm": 111, + "selected_both_correct": 6409, + "selected_both_wrong": 512 + }, + { + "budget": 0.7, + "threshold": -0.9990302324295044, + "operator": "gt", + "actual_fraction": 0.7000789265982637, + "metric": 0.9053598335366291, + "rescued": 2094, + "harmed": 626, + "missed_rescue": 131, + "avoided_harm": 103, + "selected_both_correct": 6523, + "selected_both_wrong": 514 + }, + { + "budget": 0.71, + "threshold": -0.9991169571876526, + "operator": "gt", + "actual_fraction": 0.7121331706967066, + "metric": 0.9056468393484968, + "rescued": 2103, + "harmed": 631, + "missed_rescue": 122, + "avoided_harm": 98, + "selected_both_correct": 6676, + "selected_both_wrong": 515 + }, + { + "budget": 0.72, + "threshold": -0.9991704225540161, + "operator": "gt", + "actual_fraction": 0.7201693334290019, + "metric": 0.9060055966133315, + "rescued": 2110, + "harmed": 633, + "missed_rescue": 115, + "avoided_harm": 96, + "selected_both_correct": 6779, + "selected_both_wrong": 515 + }, + { + "budget": 0.73, + "threshold": -0.9992445707321167, + "operator": "gt", + "actual_fraction": 0.7310755542799742, + "metric": 0.9062208509722322, + "rescued": 2117, + "harmed": 637, + "missed_rescue": 108, + "avoided_harm": 92, + "selected_both_correct": 6917, + "selected_both_wrong": 518 + }, + { + "budget": 0.74, + "threshold": -0.9992903470993042, + "operator": "gt", + "actual_fraction": 0.7394704742771041, + "metric": 0.9062208509722322, + "rescued": 2121, + "harmed": 641, + "missed_rescue": 104, + "avoided_harm": 88, + "selected_both_correct": 7024, + "selected_both_wrong": 520 + }, + { + "budget": 0.75, + "threshold": -0.9993333220481873, + "operator": "gt", + "actual_fraction": 0.7467173710267633, + "metric": 0.9065796082370668, + "rescued": 2127, + "harmed": 642, + "missed_rescue": 98, + "avoided_harm": 87, + "selected_both_correct": 7118, + "selected_both_wrong": 520 + }, + { + "budget": 0.76, + "threshold": -0.999373733997345, + "operator": "gt", + "actual_fraction": 0.7550405395709263, + "metric": 0.9065796082370668, + "rescued": 2129, + "harmed": 644, + "missed_rescue": 96, + "avoided_harm": 85, + "selected_both_correct": 7227, + "selected_both_wrong": 523 + }, + { + "budget": 0.77, + "threshold": -0.9994296431541443, + "operator": "gt", + "actual_fraction": 0.7659467604218986, + "metric": 0.9066513596900337, + "rescued": 2136, + "harmed": 650, + "missed_rescue": 89, + "avoided_harm": 79, + "selected_both_correct": 7362, + "selected_both_wrong": 527 + }, + { + "budget": 0.78, + "threshold": -0.999464213848114, + "operator": "gt", + "actual_fraction": 0.7743416804190285, + "metric": 0.9071536198608022, + "rescued": 2145, + "harmed": 652, + "missed_rescue": 80, + "avoided_harm": 77, + "selected_both_correct": 7467, + "selected_both_wrong": 528 + }, + { + "budget": 0.79, + "threshold": -0.9994966983795166, + "operator": "gt", + "actual_fraction": 0.7820908373394562, + "metric": 0.9071536198608022, + "rescued": 2150, + "harmed": 657, + "missed_rescue": 75, + "avoided_harm": 72, + "selected_both_correct": 7563, + "selected_both_wrong": 530 + }, + { + "budget": 0.8, + "threshold": -0.9995417594909668, + "operator": "gt", + "actual_fraction": 0.7935710698141637, + "metric": 0.9070101169548683, + "rescued": 2157, + "harmed": 666, + "missed_rescue": 68, + "avoided_harm": 63, + "selected_both_correct": 7706, + "selected_both_wrong": 531 + }, + { + "budget": 0.81, + "threshold": -0.9995827078819275, + "operator": "gt", + "actual_fraction": 0.8052665566477721, + "metric": 0.907297122766736, + "rescued": 2166, + "harmed": 671, + "missed_rescue": 59, + "avoided_harm": 58, + "selected_both_correct": 7854, + "selected_both_wrong": 532 + }, + { + "budget": 0.82, + "threshold": -0.9996140599250793, + "operator": "gt", + "actual_fraction": 0.8185405754466528, + "metric": 0.907297122766736, + "rescued": 2172, + "harmed": 677, + "missed_rescue": 53, + "avoided_harm": 52, + "selected_both_correct": 8022, + "selected_both_wrong": 537 + }, + { + "budget": 0.83, + "threshold": -0.9996429681777954, + "operator": "gt", + "actual_fraction": 0.8264332352730143, + "metric": 0.9075123771256368, + "rescued": 2176, + "harmed": 678, + "missed_rescue": 49, + "avoided_harm": 51, + "selected_both_correct": 8126, + "selected_both_wrong": 538 + }, + { + "budget": 0.84, + "threshold": -0.9996750354766846, + "operator": "gt", + "actual_fraction": 0.836837195953218, + "metric": 0.9075841285786037, + "rescued": 2179, + "harmed": 680, + "missed_rescue": 46, + "avoided_harm": 49, + "selected_both_correct": 8265, + "selected_both_wrong": 539 + }, + { + "budget": 0.85, + "threshold": -0.9997040629386902, + "operator": "gt", + "actual_fraction": 0.844299347061778, + "metric": 0.9075123771256368, + "rescued": 2183, + "harmed": 685, + "missed_rescue": 42, + "avoided_harm": 44, + "selected_both_correct": 8359, + "selected_both_wrong": 540 + }, + { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8580038745784602, + "metric": 0.9077276314845376, + "rescued": 2191, + "harmed": 690, + "missed_rescue": 34, + "avoided_harm": 39, + "selected_both_correct": 8533, + "selected_both_wrong": 544 + }, + { + "budget": 0.87, + "threshold": -0.9997695088386536, + "operator": "gt", + "actual_fraction": 0.870273373035804, + "metric": 0.9081581402023391, + "rescued": 2199, + "harmed": 692, + "missed_rescue": 26, + "avoided_harm": 37, + "selected_both_correct": 8692, + "selected_both_wrong": 546 + }, + { + "budget": 0.88, + "threshold": -0.999783456325531, + "operator": "gt", + "actual_fraction": 0.8750807203845878, + "metric": 0.9080863887493722, + "rescued": 2199, + "harmed": 693, + "missed_rescue": 26, + "avoided_harm": 36, + "selected_both_correct": 8757, + "selected_both_wrong": 547 + }, + { + "budget": 0.89, + "threshold": -0.9998205304145813, + "operator": "gt", + "actual_fraction": 0.8908660400373107, + "metric": 0.9080146372964053, + "rescued": 2202, + "harmed": 697, + "missed_rescue": 23, + "avoided_harm": 32, + "selected_both_correct": 8966, + "selected_both_wrong": 551 + }, + { + "budget": 0.9, + "threshold": -0.999841570854187, + "operator": "gt", + "actual_fraction": 0.8992609600344407, + "metric": 0.9077276314845376, + "rescued": 2204, + "harmed": 703, + "missed_rescue": 21, + "avoided_harm": 26, + "selected_both_correct": 9075, + "selected_both_wrong": 551 + }, + { + "budget": 0.91, + "threshold": -0.9998601675033569, + "operator": "gt", + "actual_fraction": 0.9091626605438761, + "metric": 0.9076558800315706, + "rescued": 2206, + "harmed": 706, + "missed_rescue": 19, + "avoided_harm": 23, + "selected_both_correct": 9206, + "selected_both_wrong": 553 + }, + { + "budget": 0.92, + "threshold": -0.9998766183853149, + "operator": "gt", + "actual_fraction": 0.9171270718232044, + "metric": 0.9077276314845376, + "rescued": 2209, + "harmed": 708, + "missed_rescue": 16, + "avoided_harm": 21, + "selected_both_correct": 9311, + "selected_both_wrong": 554 + }, + { + "budget": 0.93, + "threshold": -0.9998977184295654, + "operator": "gt", + "actual_fraction": 0.9273875295974744, + "metric": 0.9075123771256368, + "rescued": 2210, + "harmed": 712, + "missed_rescue": 15, + "avoided_harm": 17, + "selected_both_correct": 9448, + "selected_both_wrong": 555 + }, + { + "budget": 0.94, + "threshold": -0.9999151229858398, + "operator": "gt", + "actual_fraction": 0.9375044844658105, + "metric": 0.9073688742197029, + "rescued": 2213, + "harmed": 717, + "missed_rescue": 12, + "avoided_harm": 12, + "selected_both_correct": 9580, + "selected_both_wrong": 556 + }, + { + "budget": 0.95, + "threshold": -0.999931812286377, + "operator": "gt", + "actual_fraction": 0.948984716940518, + "metric": 0.9073688742197029, + "rescued": 2217, + "harmed": 721, + "missed_rescue": 8, + "avoided_harm": 8, + "selected_both_correct": 9731, + "selected_both_wrong": 557 + }, + { + "budget": 0.96, + "threshold": -0.9999452829360962, + "operator": "gt", + "actual_fraction": 0.9576666427495156, + "metric": 0.9073688742197029, + "rescued": 2217, + "harmed": 721, + "missed_rescue": 8, + "avoided_harm": 8, + "selected_both_correct": 9850, + "selected_both_wrong": 559 + }, + { + "budget": 0.97, + "threshold": -0.9999599456787109, + "operator": "gt", + "actual_fraction": 0.9709406615483963, + "metric": 0.9072253713137691, + "rescued": 2217, + "harmed": 723, + "missed_rescue": 8, + "avoided_harm": 6, + "selected_both_correct": 10033, + "selected_both_wrong": 559 + }, + { + "budget": 0.98, + "threshold": -0.9999687671661377, + "operator": "gt", + "actual_fraction": 0.9784745641099232, + "metric": 0.907297122766736, + "rescued": 2218, + "harmed": 723, + "missed_rescue": 7, + "avoided_harm": 6, + "selected_both_correct": 10136, + "selected_both_wrong": 560 + }, + { + "budget": 0.99, + "threshold": -0.9999842643737793, + "operator": "gt", + "actual_fraction": 0.98973954222573, + "metric": 0.9072253713137691, + "rescued": 2222, + "harmed": 728, + "missed_rescue": 3, + "avoided_harm": 1, + "selected_both_correct": 10282, + "selected_both_wrong": 562 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9073688742197029, + "rescued": 2225, + "harmed": 729, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 10421, + "selected_both_wrong": 562 + } + ] + }, + "learned": { + "selected_budget_index": 85, + "validation_area_0_25": 0.8359782297336007, + "validation_point": { + "budget": 0.85, + "threshold": -5.675332069396973, + "operator": "ge", + "actual_fraction": 0.8498997421942137, + "metric": 0.8994557433400172, + "rescued": 1060, + "harmed": 353, + "missed_rescue": 27, + "avoided_harm": 30, + "selected_both_correct": 4216, + "selected_both_wrong": 305 + }, + "test_point": { + "budget": 0.85, + "threshold": -5.675332069396973, + "operator": "ge", + "actual_fraction": 0.8465953935567195, + "metric": 0.906436105331133, + "rescued": 2140, + "harmed": 657, + "missed_rescue": 85, + "avoided_harm": 72, + "selected_both_correct": 8480, + "selected_both_wrong": 522 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7981953594958464, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1087, + "avoided_harm": 383, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 3.0303702354431152, + "operator": "ge", + "actual_fraction": 0.009882555141793183, + "metric": 0.8014895445431108, + "rescued": 33, + "harmed": 10, + "missed_rescue": 1054, + "avoided_harm": 373, + "selected_both_correct": 23, + "selected_both_wrong": 3 + }, + { + "budget": 0.02, + "threshold": 2.5525929927825928, + "operator": "ge", + "actual_fraction": 0.01990833572042395, + "metric": 0.8066456602692639, + "rescued": 73, + "harmed": 14, + "missed_rescue": 1014, + "avoided_harm": 369, + "selected_both_correct": 49, + "selected_both_wrong": 3 + }, + { + "budget": 0.03, + "threshold": 2.2229702472686768, + "operator": "ge", + "actual_fraction": 0.029934116299054712, + "metric": 0.8095101690060155, + "rescued": 99, + "harmed": 20, + "missed_rescue": 988, + "avoided_harm": 363, + "selected_both_correct": 82, + "selected_both_wrong": 8 + }, + { + "budget": 0.04, + "threshold": 1.9550902843475342, + "operator": "ge", + "actual_fraction": 0.039959896877685475, + "metric": 0.813806932111143, + "rescued": 131, + "harmed": 22, + "missed_rescue": 956, + "avoided_harm": 361, + "selected_both_correct": 113, + "selected_both_wrong": 13 + }, + { + "budget": 0.05, + "threshold": 1.7211925983428955, + "operator": "ge", + "actual_fraction": 0.04998567745631624, + "metric": 0.8173875680320825, + "rescued": 159, + "harmed": 25, + "missed_rescue": 928, + "avoided_harm": 358, + "selected_both_correct": 149, + "selected_both_wrong": 16 + }, + { + "budget": 0.06, + "threshold": 1.5252487659454346, + "operator": "ge", + "actual_fraction": 0.059868232598109423, + "metric": 0.8202520767688342, + "rescued": 189, + "harmed": 35, + "missed_rescue": 898, + "avoided_harm": 348, + "selected_both_correct": 175, + "selected_both_wrong": 19 + }, + { + "budget": 0.07, + "threshold": 1.363503098487854, + "operator": "ge", + "actual_fraction": 0.06989401317674018, + "metric": 0.8236894872529361, + "rescued": 215, + "harmed": 37, + "missed_rescue": 872, + "avoided_harm": 346, + "selected_both_correct": 210, + "selected_both_wrong": 26 + }, + { + "budget": 0.08, + "threshold": 1.2128210067749023, + "operator": "ge", + "actual_fraction": 0.07991979375537095, + "metric": 0.8259810942423375, + "rescued": 237, + "harmed": 43, + "missed_rescue": 850, + "avoided_harm": 340, + "selected_both_correct": 246, + "selected_both_wrong": 32 + }, + { + "budget": 0.09, + "threshold": 1.0626776218414307, + "operator": "ge", + "actual_fraction": 0.08994557433400172, + "metric": 0.8279862503580636, + "rescued": 257, + "harmed": 49, + "missed_rescue": 830, + "avoided_harm": 334, + "selected_both_correct": 285, + "selected_both_wrong": 37 + }, + { + "budget": 0.1, + "threshold": 0.9367613196372986, + "operator": "ge", + "actual_fraction": 0.09997135491263248, + "metric": 0.8315668862790031, + "rescued": 284, + "harmed": 51, + "missed_rescue": 803, + "avoided_harm": 332, + "selected_both_correct": 320, + "selected_both_wrong": 43 + }, + { + "budget": 0.11, + "threshold": 0.8210691213607788, + "operator": "ge", + "actual_fraction": 0.10999713549126325, + "metric": 0.83471784588943, + "rescued": 313, + "harmed": 58, + "missed_rescue": 774, + "avoided_harm": 325, + "selected_both_correct": 348, + "selected_both_wrong": 49 + }, + { + "budget": 0.12, + "threshold": 0.7140483856201172, + "operator": "ge", + "actual_fraction": 0.11987969063305644, + "metric": 0.8378688054998568, + "rescued": 337, + "harmed": 60, + "missed_rescue": 750, + "avoided_harm": 323, + "selected_both_correct": 386, + "selected_both_wrong": 54 + }, + { + "budget": 0.13, + "threshold": 0.6108216047286987, + "operator": "ge", + "actual_fraction": 0.1299054712116872, + "metric": 0.8405900887997708, + "rescued": 363, + "harmed": 67, + "missed_rescue": 724, + "avoided_harm": 316, + "selected_both_correct": 419, + "selected_both_wrong": 58 + }, + { + "budget": 0.14, + "threshold": 0.5261733531951904, + "operator": "ge", + "actual_fraction": 0.13993125179031796, + "metric": 0.8427384703523346, + "rescued": 384, + "harmed": 73, + "missed_rescue": 703, + "avoided_harm": 310, + "selected_both_correct": 456, + "selected_both_wrong": 64 + }, + { + "budget": 0.15, + "threshold": 0.415357381105423, + "operator": "ge", + "actual_fraction": 0.14995703236894872, + "metric": 0.844313950157548, + "rescued": 403, + "harmed": 81, + "missed_rescue": 684, + "avoided_harm": 302, + "selected_both_correct": 491, + "selected_both_wrong": 72 + }, + { + "budget": 0.16, + "threshold": 0.33500272035598755, + "operator": "ge", + "actual_fraction": 0.1599828129475795, + "metric": 0.8463191062732741, + "rescued": 420, + "harmed": 84, + "missed_rescue": 667, + "avoided_harm": 299, + "selected_both_correct": 535, + "selected_both_wrong": 78 + }, + { + "budget": 0.17, + "threshold": 0.23127245903015137, + "operator": "ge", + "actual_fraction": 0.16986536808937266, + "metric": 0.8483242623890003, + "rescued": 438, + "harmed": 88, + "missed_rescue": 649, + "avoided_harm": 295, + "selected_both_correct": 574, + "selected_both_wrong": 86 + }, + { + "budget": 0.18, + "threshold": 0.1431606113910675, + "operator": "ge", + "actual_fraction": 0.17989114866800343, + "metric": 0.8504726439415641, + "rescued": 455, + "harmed": 90, + "missed_rescue": 632, + "avoided_harm": 293, + "selected_both_correct": 616, + "selected_both_wrong": 95 + }, + { + "budget": 0.19, + "threshold": 0.05543719232082367, + "operator": "ge", + "actual_fraction": 0.1899169292466342, + "metric": 0.853910054425666, + "rescued": 481, + "harmed": 92, + "missed_rescue": 606, + "avoided_harm": 291, + "selected_both_correct": 652, + "selected_both_wrong": 101 + }, + { + "budget": 0.2, + "threshold": -0.03553308546543121, + "operator": "ge", + "actual_fraction": 0.19994270982526496, + "metric": 0.8549126324835291, + "rescued": 497, + "harmed": 101, + "missed_rescue": 590, + "avoided_harm": 282, + "selected_both_correct": 691, + "selected_both_wrong": 107 + }, + { + "budget": 0.21, + "threshold": -0.11348894238471985, + "operator": "ge", + "actual_fraction": 0.20996849040389573, + "metric": 0.8572042394729303, + "rescued": 516, + "harmed": 104, + "missed_rescue": 571, + "avoided_harm": 279, + "selected_both_correct": 733, + "selected_both_wrong": 113 + }, + { + "budget": 0.22, + "threshold": -0.20718136429786682, + "operator": "ge", + "actual_fraction": 0.2199942709825265, + "metric": 0.8594958464623317, + "rescued": 537, + "harmed": 109, + "missed_rescue": 550, + "avoided_harm": 274, + "selected_both_correct": 772, + "selected_both_wrong": 118 + }, + { + "budget": 0.23, + "threshold": -0.28485560417175293, + "operator": "ge", + "actual_fraction": 0.22987682612431967, + "metric": 0.861787453451733, + "rescued": 559, + "harmed": 115, + "missed_rescue": 528, + "avoided_harm": 268, + "selected_both_correct": 811, + "selected_both_wrong": 120 + }, + { + "budget": 0.24, + "threshold": -0.35161370038986206, + "operator": "ge", + "actual_fraction": 0.23990260670295044, + "metric": 0.8640790604411344, + "rescued": 577, + "harmed": 117, + "missed_rescue": 510, + "avoided_harm": 266, + "selected_both_correct": 854, + "selected_both_wrong": 127 + }, + { + "budget": 0.25, + "threshold": -0.42771607637405396, + "operator": "ge", + "actual_fraction": 0.2499283872815812, + "metric": 0.865224863935835, + "rescued": 586, + "harmed": 118, + "missed_rescue": 501, + "avoided_harm": 265, + "selected_both_correct": 908, + "selected_both_wrong": 133 + }, + { + "budget": 0.26, + "threshold": -0.5070093870162964, + "operator": "ge", + "actual_fraction": 0.25995416786021197, + "metric": 0.8675164709252363, + "rescued": 607, + "harmed": 123, + "missed_rescue": 480, + "avoided_harm": 260, + "selected_both_correct": 948, + "selected_both_wrong": 137 + }, + { + "budget": 0.27, + "threshold": -0.5965517163276672, + "operator": "ge", + "actual_fraction": 0.26997994843884277, + "metric": 0.8683758235462619, + "rescued": 621, + "harmed": 131, + "missed_rescue": 466, + "avoided_harm": 252, + "selected_both_correct": 993, + "selected_both_wrong": 140 + }, + { + "budget": 0.28, + "threshold": -0.6840324401855469, + "operator": "ge", + "actual_fraction": 0.2798625035806359, + "metric": 0.8698080779146377, + "rescued": 638, + "harmed": 138, + "missed_rescue": 449, + "avoided_harm": 245, + "selected_both_correct": 1029, + "selected_both_wrong": 149 + }, + { + "budget": 0.29, + "threshold": -0.7701197266578674, + "operator": "ge", + "actual_fraction": 0.2898882841592667, + "metric": 0.8715267831566886, + "rescued": 654, + "harmed": 142, + "missed_rescue": 433, + "avoided_harm": 241, + "selected_both_correct": 1073, + "selected_both_wrong": 155 + }, + { + "budget": 0.3, + "threshold": -0.8396241068840027, + "operator": "ge", + "actual_fraction": 0.29991406473789745, + "metric": 0.8723861357777141, + "rescued": 665, + "harmed": 147, + "missed_rescue": 422, + "avoided_harm": 236, + "selected_both_correct": 1121, + "selected_both_wrong": 161 + }, + { + "budget": 0.31, + "threshold": -0.9167145490646362, + "operator": "ge", + "actual_fraction": 0.30993984531652824, + "metric": 0.8726725866513892, + "rescued": 672, + "harmed": 152, + "missed_rescue": 415, + "avoided_harm": 231, + "selected_both_correct": 1175, + "selected_both_wrong": 165 + }, + { + "budget": 0.32, + "threshold": -0.9918558597564697, + "operator": "ge", + "actual_fraction": 0.319965625895159, + "metric": 0.8729590375250644, + "rescued": 680, + "harmed": 158, + "missed_rescue": 407, + "avoided_harm": 225, + "selected_both_correct": 1221, + "selected_both_wrong": 175 + }, + { + "budget": 0.33, + "threshold": -1.0728092193603516, + "operator": "ge", + "actual_fraction": 0.3299914064737898, + "metric": 0.8739616155829275, + "rescued": 695, + "harmed": 166, + "missed_rescue": 392, + "avoided_harm": 217, + "selected_both_correct": 1263, + "selected_both_wrong": 180 + }, + { + "budget": 0.34, + "threshold": -1.1429637670516968, + "operator": "ge", + "actual_fraction": 0.3398739616155829, + "metric": 0.8746777427671154, + "rescued": 705, + "harmed": 171, + "missed_rescue": 382, + "avoided_harm": 212, + "selected_both_correct": 1312, + "selected_both_wrong": 185 + }, + { + "budget": 0.35, + "threshold": -1.2190481424331665, + "operator": "ge", + "actual_fraction": 0.3498997421942137, + "metric": 0.8752506445144658, + "rescued": 719, + "harmed": 181, + "missed_rescue": 368, + "avoided_harm": 202, + "selected_both_correct": 1355, + "selected_both_wrong": 188 + }, + { + "budget": 0.36, + "threshold": -1.3103632926940918, + "operator": "ge", + "actual_fraction": 0.35992552277284445, + "metric": 0.8772558006301919, + "rescued": 738, + "harmed": 186, + "missed_rescue": 349, + "avoided_harm": 197, + "selected_both_correct": 1400, + "selected_both_wrong": 189 + }, + { + "budget": 0.37, + "threshold": -1.403131365776062, + "operator": "ge", + "actual_fraction": 0.36995130335147525, + "metric": 0.8785448295617302, + "rescued": 752, + "harmed": 191, + "missed_rescue": 335, + "avoided_harm": 192, + "selected_both_correct": 1449, + "selected_both_wrong": 191 + }, + { + "budget": 0.38, + "threshold": -1.4784197807312012, + "operator": "ge", + "actual_fraction": 0.379977083930106, + "metric": 0.8791177313090804, + "rescued": 762, + "harmed": 197, + "missed_rescue": 325, + "avoided_harm": 186, + "selected_both_correct": 1501, + "selected_both_wrong": 193 + }, + { + "budget": 0.39, + "threshold": -1.57342529296875, + "operator": "ge", + "actual_fraction": 0.3898596390718992, + "metric": 0.8802635348037812, + "rescued": 773, + "harmed": 200, + "missed_rescue": 314, + "avoided_harm": 183, + "selected_both_correct": 1549, + "selected_both_wrong": 200 + }, + { + "budget": 0.4, + "threshold": -1.6548807621002197, + "operator": "ge", + "actual_fraction": 0.3998854196505299, + "metric": 0.8818390146089946, + "rescued": 785, + "harmed": 201, + "missed_rescue": 302, + "avoided_harm": 182, + "selected_both_correct": 1601, + "selected_both_wrong": 205 + }, + { + "budget": 0.41, + "threshold": -1.7329905033111572, + "operator": "ge", + "actual_fraction": 0.4099112002291607, + "metric": 0.8835577198510456, + "rescued": 800, + "harmed": 204, + "missed_rescue": 287, + "avoided_harm": 179, + "selected_both_correct": 1651, + "selected_both_wrong": 207 + }, + { + "budget": 0.42, + "threshold": -1.8175678253173828, + "operator": "ge", + "actual_fraction": 0.41993698080779146, + "metric": 0.8838441707247207, + "rescued": 809, + "harmed": 211, + "missed_rescue": 278, + "avoided_harm": 172, + "selected_both_correct": 1704, + "selected_both_wrong": 208 + }, + { + "budget": 0.43, + "threshold": -1.8905004262924194, + "operator": "ge", + "actual_fraction": 0.42996276138642225, + "metric": 0.8839873961615583, + "rescued": 817, + "harmed": 218, + "missed_rescue": 270, + "avoided_harm": 165, + "selected_both_correct": 1757, + "selected_both_wrong": 210 + }, + { + "budget": 0.44, + "threshold": -1.9868656396865845, + "operator": "ge", + "actual_fraction": 0.439988541965053, + "metric": 0.8847035233457462, + "rescued": 826, + "harmed": 222, + "missed_rescue": 261, + "avoided_harm": 161, + "selected_both_correct": 1812, + "selected_both_wrong": 212 + }, + { + "budget": 0.45, + "threshold": -2.078063726425171, + "operator": "ge", + "actual_fraction": 0.4498710971068462, + "metric": 0.8855628759667717, + "rescued": 835, + "harmed": 225, + "missed_rescue": 252, + "avoided_harm": 158, + "selected_both_correct": 1865, + "selected_both_wrong": 216 + }, + { + "budget": 0.46, + "threshold": -2.1543872356414795, + "operator": "ge", + "actual_fraction": 0.45989687768547693, + "metric": 0.8855628759667717, + "rescued": 842, + "harmed": 232, + "missed_rescue": 245, + "avoided_harm": 151, + "selected_both_correct": 1917, + "selected_both_wrong": 220 + }, + { + "budget": 0.47, + "threshold": -2.22452712059021, + "operator": "ge", + "actual_fraction": 0.46992265826410773, + "metric": 0.8867086794614724, + "rescued": 850, + "harmed": 232, + "missed_rescue": 237, + "avoided_harm": 151, + "selected_both_correct": 1974, + "selected_both_wrong": 225 + }, + { + "budget": 0.48, + "threshold": -2.2989747524261475, + "operator": "ge", + "actual_fraction": 0.47994843884273847, + "metric": 0.8868519048983099, + "rescued": 857, + "harmed": 238, + "missed_rescue": 230, + "avoided_harm": 145, + "selected_both_correct": 2028, + "selected_both_wrong": 228 + }, + { + "budget": 0.49, + "threshold": -2.402249336242676, + "operator": "ge", + "actual_fraction": 0.48997421942136926, + "metric": 0.8872815812088227, + "rescued": 864, + "harmed": 242, + "missed_rescue": 223, + "avoided_harm": 141, + "selected_both_correct": 2082, + "selected_both_wrong": 233 + }, + { + "budget": 0.5, + "threshold": -2.477130174636841, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.8884273847035233, + "rescued": 875, + "harmed": 245, + "missed_rescue": 212, + "avoided_harm": 138, + "selected_both_correct": 2136, + "selected_both_wrong": 235 + }, + { + "budget": 0.51, + "threshold": -2.5623269081115723, + "operator": "ge", + "actual_fraction": 0.5098825551417931, + "metric": 0.8894299627613864, + "rescued": 887, + "harmed": 250, + "missed_rescue": 200, + "avoided_harm": 133, + "selected_both_correct": 2186, + "selected_both_wrong": 237 + }, + { + "budget": 0.52, + "threshold": -2.6437463760375977, + "operator": "ge", + "actual_fraction": 0.5199083357204239, + "metric": 0.8897164136350616, + "rescued": 895, + "harmed": 256, + "missed_rescue": 192, + "avoided_harm": 127, + "selected_both_correct": 2240, + "selected_both_wrong": 239 + }, + { + "budget": 0.53, + "threshold": -2.7279462814331055, + "operator": "ge", + "actual_fraction": 0.5299341162990547, + "metric": 0.8910054425665999, + "rescued": 907, + "harmed": 259, + "missed_rescue": 180, + "avoided_harm": 124, + "selected_both_correct": 2293, + "selected_both_wrong": 241 + }, + { + "budget": 0.54, + "threshold": -2.832216262817383, + "operator": "ge", + "actual_fraction": 0.5399598968776855, + "metric": 0.8911486680034374, + "rescued": 912, + "harmed": 263, + "missed_rescue": 175, + "avoided_harm": 120, + "selected_both_correct": 2353, + "selected_both_wrong": 242 + }, + { + "budget": 0.55, + "threshold": -2.9212465286254883, + "operator": "ge", + "actual_fraction": 0.5499856774563162, + "metric": 0.8910054425665999, + "rescued": 920, + "harmed": 272, + "missed_rescue": 167, + "avoided_harm": 111, + "selected_both_correct": 2402, + "selected_both_wrong": 246 + }, + { + "budget": 0.56, + "threshold": -3.0128684043884277, + "operator": "ge", + "actual_fraction": 0.5598682325981095, + "metric": 0.8918647951876253, + "rescued": 928, + "harmed": 274, + "missed_rescue": 159, + "avoided_harm": 109, + "selected_both_correct": 2455, + "selected_both_wrong": 252 + }, + { + "budget": 0.57, + "threshold": -3.1058738231658936, + "operator": "ge", + "actual_fraction": 0.5698940131767402, + "metric": 0.8928673732454884, + "rescued": 937, + "harmed": 276, + "missed_rescue": 150, + "avoided_harm": 107, + "selected_both_correct": 2508, + "selected_both_wrong": 258 + }, + { + "budget": 0.58, + "threshold": -3.1683619022369385, + "operator": "ge", + "actual_fraction": 0.579919793755371, + "metric": 0.8940131767401891, + "rescued": 947, + "harmed": 278, + "missed_rescue": 140, + "avoided_harm": 105, + "selected_both_correct": 2563, + "selected_both_wrong": 261 + }, + { + "budget": 0.59, + "threshold": -3.2430496215820312, + "operator": "ge", + "actual_fraction": 0.5899455743340017, + "metric": 0.8944428530507018, + "rescued": 951, + "harmed": 279, + "missed_rescue": 136, + "avoided_harm": 104, + "selected_both_correct": 2624, + "selected_both_wrong": 265 + }, + { + "budget": 0.6, + "threshold": -3.325876235961914, + "operator": "ge", + "actual_fraction": 0.5999713549126325, + "metric": 0.8942996276138642, + "rescued": 955, + "harmed": 284, + "missed_rescue": 132, + "avoided_harm": 99, + "selected_both_correct": 2684, + "selected_both_wrong": 266 + }, + { + "budget": 0.61, + "threshold": -3.407817840576172, + "operator": "ge", + "actual_fraction": 0.6099971354912632, + "metric": 0.8948725293612145, + "rescued": 961, + "harmed": 286, + "missed_rescue": 126, + "avoided_harm": 97, + "selected_both_correct": 2744, + "selected_both_wrong": 268 + }, + { + "budget": 0.62, + "threshold": -3.487114191055298, + "operator": "ge", + "actual_fraction": 0.6198796906330565, + "metric": 0.8953022056717272, + "rescued": 968, + "harmed": 290, + "missed_rescue": 119, + "avoided_harm": 93, + "selected_both_correct": 2800, + "selected_both_wrong": 270 + }, + { + "budget": 0.63, + "threshold": -3.5880866050720215, + "operator": "ge", + "actual_fraction": 0.6299054712116872, + "metric": 0.8953022056717272, + "rescued": 971, + "harmed": 293, + "missed_rescue": 116, + "avoided_harm": 90, + "selected_both_correct": 2863, + "selected_both_wrong": 271 + }, + { + "budget": 0.64, + "threshold": -3.671746253967285, + "operator": "ge", + "actual_fraction": 0.639931251790318, + "metric": 0.8951589802348897, + "rescued": 973, + "harmed": 296, + "missed_rescue": 114, + "avoided_harm": 87, + "selected_both_correct": 2928, + "selected_both_wrong": 271 + }, + { + "budget": 0.65, + "threshold": -3.759582281112671, + "operator": "ge", + "actual_fraction": 0.6499570323689488, + "metric": 0.8954454311085649, + "rescued": 978, + "harmed": 299, + "missed_rescue": 109, + "avoided_harm": 84, + "selected_both_correct": 2989, + "selected_both_wrong": 272 + }, + { + "budget": 0.66, + "threshold": -3.850314140319824, + "operator": "ge", + "actual_fraction": 0.6599828129475795, + "metric": 0.8955886565454024, + "rescued": 984, + "harmed": 304, + "missed_rescue": 103, + "avoided_harm": 79, + "selected_both_correct": 3047, + "selected_both_wrong": 273 + }, + { + "budget": 0.67, + "threshold": -3.9420368671417236, + "operator": "ge", + "actual_fraction": 0.6698653680893727, + "metric": 0.8958751074190776, + "rescued": 988, + "harmed": 306, + "missed_rescue": 99, + "avoided_harm": 77, + "selected_both_correct": 3109, + "selected_both_wrong": 274 + }, + { + "budget": 0.68, + "threshold": -4.02707052230835, + "operator": "ge", + "actual_fraction": 0.6798911486680035, + "metric": 0.8967344600401032, + "rescued": 996, + "harmed": 308, + "missed_rescue": 91, + "avoided_harm": 75, + "selected_both_correct": 3166, + "selected_both_wrong": 277 + }, + { + "budget": 0.69, + "threshold": -4.099420547485352, + "operator": "ge", + "actual_fraction": 0.6899169292466342, + "metric": 0.896448009166428, + "rescued": 998, + "harmed": 312, + "missed_rescue": 89, + "avoided_harm": 71, + "selected_both_correct": 3227, + "selected_both_wrong": 280 + }, + { + "budget": 0.7, + "threshold": -4.196893692016602, + "operator": "ge", + "actual_fraction": 0.699942709825265, + "metric": 0.8965912346032655, + "rescued": 1003, + "harmed": 316, + "missed_rescue": 84, + "avoided_harm": 67, + "selected_both_correct": 3288, + "selected_both_wrong": 280 + }, + { + "budget": 0.71, + "threshold": -4.279935359954834, + "operator": "ge", + "actual_fraction": 0.7099684904038958, + "metric": 0.8968776854769407, + "rescued": 1008, + "harmed": 319, + "missed_rescue": 79, + "avoided_harm": 64, + "selected_both_correct": 3349, + "selected_both_wrong": 281 + }, + { + "budget": 0.72, + "threshold": -4.365882873535156, + "operator": "ge", + "actual_fraction": 0.7199942709825266, + "metric": 0.8970209109137783, + "rescued": 1013, + "harmed": 323, + "missed_rescue": 74, + "avoided_harm": 60, + "selected_both_correct": 3406, + "selected_both_wrong": 285 + }, + { + "budget": 0.73, + "threshold": -4.45527982711792, + "operator": "ge", + "actual_fraction": 0.7298768261243197, + "metric": 0.897450587224291, + "rescued": 1018, + "harmed": 325, + "missed_rescue": 69, + "avoided_harm": 58, + "selected_both_correct": 3465, + "selected_both_wrong": 288 + }, + { + "budget": 0.74, + "threshold": -4.553353786468506, + "operator": "ge", + "actual_fraction": 0.7399026067029505, + "metric": 0.8971641363506159, + "rescued": 1019, + "harmed": 328, + "missed_rescue": 68, + "avoided_harm": 55, + "selected_both_correct": 3529, + "selected_both_wrong": 290 + }, + { + "budget": 0.75, + "threshold": -4.638945579528809, + "operator": "ge", + "actual_fraction": 0.7499283872815812, + "metric": 0.8975938126611286, + "rescued": 1024, + "harmed": 330, + "missed_rescue": 63, + "avoided_harm": 53, + "selected_both_correct": 3591, + "selected_both_wrong": 291 + }, + { + "budget": 0.76, + "threshold": -4.725781440734863, + "operator": "ge", + "actual_fraction": 0.759954167860212, + "metric": 0.897450587224291, + "rescued": 1024, + "harmed": 331, + "missed_rescue": 63, + "avoided_harm": 52, + "selected_both_correct": 3657, + "selected_both_wrong": 294 + }, + { + "budget": 0.77, + "threshold": -4.809637069702148, + "operator": "ge", + "actual_fraction": 0.7699799484388428, + "metric": 0.8973073617874534, + "rescued": 1026, + "harmed": 334, + "missed_rescue": 61, + "avoided_harm": 49, + "selected_both_correct": 3721, + "selected_both_wrong": 295 + }, + { + "budget": 0.78, + "threshold": -4.926034927368164, + "operator": "ge", + "actual_fraction": 0.7798625035806359, + "metric": 0.8975938126611286, + "rescued": 1030, + "harmed": 336, + "missed_rescue": 57, + "avoided_harm": 47, + "selected_both_correct": 3782, + "selected_both_wrong": 297 + }, + { + "budget": 0.79, + "threshold": -5.015888690948486, + "operator": "ge", + "actual_fraction": 0.7898882841592667, + "metric": 0.897450587224291, + "rescued": 1034, + "harmed": 341, + "missed_rescue": 53, + "avoided_harm": 42, + "selected_both_correct": 3842, + "selected_both_wrong": 298 + }, + { + "budget": 0.8, + "threshold": -5.120303630828857, + "operator": "ge", + "actual_fraction": 0.7999140647378975, + "metric": 0.8977370380979662, + "rescued": 1039, + "harmed": 344, + "missed_rescue": 48, + "avoided_harm": 39, + "selected_both_correct": 3903, + "selected_both_wrong": 299 + }, + { + "budget": 0.81, + "threshold": -5.253293037414551, + "operator": "ge", + "actual_fraction": 0.8099398453165282, + "metric": 0.8985963907189917, + "rescued": 1045, + "harmed": 344, + "missed_rescue": 42, + "avoided_harm": 39, + "selected_both_correct": 3965, + "selected_both_wrong": 301 + }, + { + "budget": 0.82, + "threshold": -5.367966175079346, + "operator": "ge", + "actual_fraction": 0.819965625895159, + "metric": 0.8988828415926668, + "rescued": 1048, + "harmed": 345, + "missed_rescue": 39, + "avoided_harm": 38, + "selected_both_correct": 4031, + "selected_both_wrong": 301 + }, + { + "budget": 0.83, + "threshold": -5.470327854156494, + "operator": "ge", + "actual_fraction": 0.8299914064737898, + "metric": 0.8985963907189917, + "rescued": 1049, + "harmed": 348, + "missed_rescue": 38, + "avoided_harm": 35, + "selected_both_correct": 4094, + "selected_both_wrong": 304 + }, + { + "budget": 0.84, + "threshold": -5.563735008239746, + "operator": "ge", + "actual_fraction": 0.8398739616155829, + "metric": 0.899169292466342, + "rescued": 1056, + "harmed": 351, + "missed_rescue": 31, + "avoided_harm": 32, + "selected_both_correct": 4152, + "selected_both_wrong": 305 + }, + { + "budget": 0.85, + "threshold": -5.675332069396973, + "operator": "ge", + "actual_fraction": 0.8498997421942137, + "metric": 0.8994557433400172, + "rescued": 1060, + "harmed": 353, + "missed_rescue": 27, + "avoided_harm": 30, + "selected_both_correct": 4216, + "selected_both_wrong": 305 + }, + { + "budget": 0.86, + "threshold": -5.787599086761475, + "operator": "ge", + "actual_fraction": 0.8599255227728445, + "metric": 0.8994557433400172, + "rescued": 1064, + "harmed": 357, + "missed_rescue": 23, + "avoided_harm": 26, + "selected_both_correct": 4276, + "selected_both_wrong": 307 + }, + { + "budget": 0.87, + "threshold": -5.91396951675415, + "operator": "ge", + "actual_fraction": 0.8699513033514752, + "metric": 0.899169292466342, + "rescued": 1064, + "harmed": 359, + "missed_rescue": 23, + "avoided_harm": 24, + "selected_both_correct": 4343, + "selected_both_wrong": 308 + }, + { + "budget": 0.88, + "threshold": -6.055293083190918, + "operator": "ge", + "actual_fraction": 0.879977083930106, + "metric": 0.8993125179031796, + "rescued": 1065, + "harmed": 359, + "missed_rescue": 22, + "avoided_harm": 24, + "selected_both_correct": 4411, + "selected_both_wrong": 309 + }, + { + "budget": 0.89, + "threshold": -6.180037975311279, + "operator": "ge", + "actual_fraction": 0.8898596390718991, + "metric": 0.8985963907189917, + "rescued": 1069, + "harmed": 368, + "missed_rescue": 18, + "avoided_harm": 15, + "selected_both_correct": 4465, + "selected_both_wrong": 311 + }, + { + "budget": 0.9, + "threshold": -6.315410614013672, + "operator": "ge", + "actual_fraction": 0.8998854196505299, + "metric": 0.8988828415926668, + "rescued": 1072, + "harmed": 369, + "missed_rescue": 15, + "avoided_harm": 14, + "selected_both_correct": 4529, + "selected_both_wrong": 313 + }, + { + "budget": 0.91, + "threshold": -6.488847732543945, + "operator": "ge", + "actual_fraction": 0.9099112002291607, + "metric": 0.899169292466342, + "rescued": 1074, + "harmed": 369, + "missed_rescue": 13, + "avoided_harm": 14, + "selected_both_correct": 4597, + "selected_both_wrong": 313 + }, + { + "budget": 0.92, + "threshold": -6.6685333251953125, + "operator": "ge", + "actual_fraction": 0.9199369808077915, + "metric": 0.8990260670295045, + "rescued": 1077, + "harmed": 373, + "missed_rescue": 10, + "avoided_harm": 10, + "selected_both_correct": 4660, + "selected_both_wrong": 313 + }, + { + "budget": 0.93, + "threshold": -6.833990097045898, + "operator": "ge", + "actual_fraction": 0.9299627613864222, + "metric": 0.8994557433400172, + "rescued": 1081, + "harmed": 374, + "missed_rescue": 6, + "avoided_harm": 9, + "selected_both_correct": 4724, + "selected_both_wrong": 314 + }, + { + "budget": 0.94, + "threshold": -7.018073558807373, + "operator": "ge", + "actual_fraction": 0.939988541965053, + "metric": 0.8990260670295045, + "rescued": 1081, + "harmed": 377, + "missed_rescue": 6, + "avoided_harm": 6, + "selected_both_correct": 4789, + "selected_both_wrong": 316 + }, + { + "budget": 0.95, + "threshold": -7.223591327667236, + "operator": "ge", + "actual_fraction": 0.9498710971068461, + "metric": 0.8990260670295045, + "rescued": 1083, + "harmed": 379, + "missed_rescue": 4, + "avoided_harm": 4, + "selected_both_correct": 4854, + "selected_both_wrong": 316 + }, + { + "budget": 0.96, + "threshold": -7.463536262512207, + "operator": "ge", + "actual_fraction": 0.9598968776854769, + "metric": 0.899169292466342, + "rescued": 1084, + "harmed": 379, + "missed_rescue": 3, + "avoided_harm": 4, + "selected_both_correct": 4923, + "selected_both_wrong": 316 + }, + { + "budget": 0.97, + "threshold": -7.692946434020996, + "operator": "ge", + "actual_fraction": 0.9699226582641077, + "metric": 0.8993125179031796, + "rescued": 1086, + "harmed": 380, + "missed_rescue": 1, + "avoided_harm": 3, + "selected_both_correct": 4988, + "selected_both_wrong": 318 + }, + { + "budget": 0.98, + "threshold": -8.048358917236328, + "operator": "ge", + "actual_fraction": 0.9799484388427385, + "metric": 0.8993125179031796, + "rescued": 1087, + "harmed": 381, + "missed_rescue": 0, + "avoided_harm": 2, + "selected_both_correct": 5055, + "selected_both_wrong": 319 + }, + { + "budget": 0.99, + "threshold": -8.652192115783691, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.899169292466342, + "rescued": 1087, + "harmed": 382, + "missed_rescue": 0, + "avoided_harm": 1, + "selected_both_correct": 5122, + "selected_both_wrong": 321 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8990260670295045, + "rescued": 1087, + "harmed": 383, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 5190, + "selected_both_wrong": 322 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.8000287005811868, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2225, + "avoided_harm": 729, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 3.0303702354431152, + "operator": "ge", + "actual_fraction": 0.01169548683360838, + "metric": 0.8064863313482098, + "rescued": 96, + "harmed": 6, + "missed_rescue": 2129, + "avoided_harm": 723, + "selected_both_correct": 55, + "selected_both_wrong": 6 + }, + { + "budget": 0.02, + "threshold": 2.5525929927825928, + "operator": "ge", + "actual_fraction": 0.022458204778646768, + "metric": 0.8114371816029274, + "rescued": 174, + "harmed": 15, + "missed_rescue": 2051, + "avoided_harm": 714, + "selected_both_correct": 107, + "selected_both_wrong": 17 + }, + { + "budget": 0.03, + "threshold": 2.2229702472686768, + "operator": "ge", + "actual_fraction": 0.03322092272368515, + "metric": 0.8166032862165459, + "rescued": 251, + "harmed": 20, + "missed_rescue": 1974, + "avoided_harm": 709, + "selected_both_correct": 165, + "selected_both_wrong": 27 + }, + { + "budget": 0.04, + "threshold": 1.9550902843475342, + "operator": "ge", + "actual_fraction": 0.04326612613905432, + "metric": 0.8208366219415943, + "rescued": 316, + "harmed": 26, + "missed_rescue": 1909, + "avoided_harm": 703, + "selected_both_correct": 231, + "selected_both_wrong": 30 + }, + { + "budget": 0.05, + "threshold": 1.7211925983428955, + "operator": "ge", + "actual_fraction": 0.05438760134892732, + "metric": 0.8244241945899404, + "rescued": 376, + "harmed": 36, + "missed_rescue": 1849, + "avoided_harm": 693, + "selected_both_correct": 305, + "selected_both_wrong": 41 + }, + { + "budget": 0.06, + "threshold": 1.5252487659454346, + "operator": "ge", + "actual_fraction": 0.06543732510583339, + "metric": 0.8272225012556504, + "rescued": 429, + "harmed": 50, + "missed_rescue": 1796, + "avoided_harm": 679, + "selected_both_correct": 382, + "selected_both_wrong": 51 + }, + { + "budget": 0.07, + "threshold": 1.363503098487854, + "operator": "ge", + "actual_fraction": 0.07541077706823564, + "metric": 0.8300925593743274, + "rescued": 479, + "harmed": 60, + "missed_rescue": 1746, + "avoided_harm": 669, + "selected_both_correct": 451, + "selected_both_wrong": 61 + }, + { + "budget": 0.08, + "threshold": 1.2128210067749023, + "operator": "ge", + "actual_fraction": 0.0867475066370094, + "metric": 0.8337518834756403, + "rescued": 541, + "harmed": 71, + "missed_rescue": 1684, + "avoided_harm": 658, + "selected_both_correct": 528, + "selected_both_wrong": 69 + }, + { + "budget": 0.09, + "threshold": 1.0626776218414307, + "operator": "ge", + "actual_fraction": 0.09636220133457703, + "metric": 0.8358326756116812, + "rescued": 580, + "harmed": 81, + "missed_rescue": 1645, + "avoided_harm": 648, + "selected_both_correct": 603, + "selected_both_wrong": 79 + }, + { + "budget": 0.1, + "threshold": 0.9367613196372986, + "operator": "ge", + "actual_fraction": 0.1074119250914831, + "metric": 0.8387027337303581, + "rescued": 631, + "harmed": 92, + "missed_rescue": 1594, + "avoided_harm": 637, + "selected_both_correct": 680, + "selected_both_wrong": 94 + }, + { + "budget": 0.11, + "threshold": 0.8210691213607788, + "operator": "ge", + "actual_fraction": 0.11609385090048073, + "metric": 0.8417880462079357, + "rescued": 679, + "harmed": 97, + "missed_rescue": 1546, + "avoided_harm": 632, + "selected_both_correct": 741, + "selected_both_wrong": 101 + }, + { + "budget": 0.12, + "threshold": 0.7140483856201172, + "operator": "ge", + "actual_fraction": 0.12592379995694913, + "metric": 0.8447298557795795, + "rescued": 727, + "harmed": 104, + "missed_rescue": 1498, + "avoided_harm": 625, + "selected_both_correct": 819, + "selected_both_wrong": 105 + }, + { + "budget": 0.13, + "threshold": 0.6108216047286987, + "operator": "ge", + "actual_fraction": 0.1353949917485829, + "metric": 0.8471694051804549, + "rescued": 770, + "harmed": 113, + "missed_rescue": 1455, + "avoided_harm": 616, + "selected_both_correct": 889, + "selected_both_wrong": 115 + }, + { + "budget": 0.14, + "threshold": 0.5261733531951904, + "operator": "ge", + "actual_fraction": 0.144579177728349, + "metric": 0.8491066944105619, + "rescued": 804, + "harmed": 120, + "missed_rescue": 1421, + "avoided_harm": 609, + "selected_both_correct": 966, + "selected_both_wrong": 125 + }, + { + "budget": 0.15, + "threshold": 0.415357381105423, + "operator": "ge", + "actual_fraction": 0.15498313840855277, + "metric": 0.851761498170338, + "rescued": 850, + "harmed": 129, + "missed_rescue": 1375, + "avoided_harm": 600, + "selected_both_correct": 1047, + "selected_both_wrong": 134 + }, + { + "budget": 0.16, + "threshold": 0.33500272035598755, + "operator": "ge", + "actual_fraction": 0.16208653225227812, + "metric": 0.8531965272296764, + "rescued": 879, + "harmed": 138, + "missed_rescue": 1346, + "avoided_harm": 591, + "selected_both_correct": 1103, + "selected_both_wrong": 139 + }, + { + "budget": 0.17, + "threshold": 0.23127245903015137, + "operator": "ge", + "actual_fraction": 0.17292100165028343, + "metric": 0.8559948338953863, + "rescued": 929, + "harmed": 149, + "missed_rescue": 1296, + "avoided_harm": 580, + "selected_both_correct": 1185, + "selected_both_wrong": 147 + }, + { + "budget": 0.18, + "threshold": 0.1431606113910675, + "operator": "ge", + "actual_fraction": 0.18289445361268566, + "metric": 0.8580038745784602, + "rescued": 969, + "harmed": 161, + "missed_rescue": 1256, + "avoided_harm": 568, + "selected_both_correct": 1259, + "selected_both_wrong": 160 + }, + { + "budget": 0.19, + "threshold": 0.05543719232082367, + "operator": "ge", + "actual_fraction": 0.19344191719882328, + "metric": 0.8606586783382364, + "rescued": 1015, + "harmed": 170, + "missed_rescue": 1210, + "avoided_harm": 559, + "selected_both_correct": 1345, + "selected_both_wrong": 166 + }, + { + "budget": 0.2, + "threshold": -0.03553308546543121, + "operator": "ge", + "actual_fraction": 0.20377412642606013, + "metric": 0.8624524646624094, + "rescued": 1051, + "harmed": 181, + "missed_rescue": 1174, + "avoided_harm": 548, + "selected_both_correct": 1434, + "selected_both_wrong": 174 + }, + { + "budget": 0.21, + "threshold": -0.11348894238471985, + "operator": "ge", + "actual_fraction": 0.21274305804692545, + "metric": 0.8646767597043841, + "rescued": 1090, + "harmed": 189, + "missed_rescue": 1135, + "avoided_harm": 540, + "selected_both_correct": 1504, + "selected_both_wrong": 182 + }, + { + "budget": 0.22, + "threshold": -0.20718136429786682, + "operator": "ge", + "actual_fraction": 0.2227882614622946, + "metric": 0.8660400373107555, + "rescued": 1120, + "harmed": 200, + "missed_rescue": 1105, + "avoided_harm": 529, + "selected_both_correct": 1591, + "selected_both_wrong": 194 + }, + { + "budget": 0.23, + "threshold": -0.28485560417175293, + "operator": "ge", + "actual_fraction": 0.23175719308315992, + "metric": 0.8680490779938294, + "rescued": 1155, + "harmed": 207, + "missed_rescue": 1070, + "avoided_harm": 522, + "selected_both_correct": 1662, + "selected_both_wrong": 206 + }, + { + "budget": 0.24, + "threshold": -0.35161370038986206, + "operator": "ge", + "actual_fraction": 0.2408696276099591, + "metric": 0.8698428643180024, + "rescued": 1188, + "harmed": 215, + "missed_rescue": 1037, + "avoided_harm": 514, + "selected_both_correct": 1738, + "selected_both_wrong": 216 + }, + { + "budget": 0.25, + "threshold": -0.42771607637405396, + "operator": "ge", + "actual_fraction": 0.2491210447011552, + "metric": 0.8709191361125063, + "rescued": 1216, + "harmed": 228, + "missed_rescue": 1009, + "avoided_harm": 501, + "selected_both_correct": 1807, + "selected_both_wrong": 221 + }, + { + "budget": 0.26, + "threshold": -0.5070093870162964, + "operator": "ge", + "actual_fraction": 0.25938150247542513, + "metric": 0.8726411709837124, + "rescued": 1249, + "harmed": 237, + "missed_rescue": 976, + "avoided_harm": 492, + "selected_both_correct": 1898, + "selected_both_wrong": 231 + }, + { + "budget": 0.27, + "threshold": -0.5965517163276672, + "operator": "ge", + "actual_fraction": 0.2707899834971658, + "metric": 0.8744349573078855, + "rescued": 1283, + "harmed": 246, + "missed_rescue": 942, + "avoided_harm": 483, + "selected_both_correct": 2006, + "selected_both_wrong": 239 + }, + { + "budget": 0.28, + "threshold": -0.6840324401855469, + "operator": "ge", + "actual_fraction": 0.28112219272440264, + "metric": 0.8760852407261247, + "rescued": 1319, + "harmed": 259, + "missed_rescue": 906, + "avoided_harm": 470, + "selected_both_correct": 2093, + "selected_both_wrong": 247 + }, + { + "budget": 0.29, + "threshold": -0.7701197266578674, + "operator": "ge", + "actual_fraction": 0.2930329339169118, + "metric": 0.8778072755973308, + "rescued": 1355, + "harmed": 271, + "missed_rescue": 870, + "avoided_harm": 458, + "selected_both_correct": 2198, + "selected_both_wrong": 260 + }, + { + "budget": 0.3, + "threshold": -0.8396241068840027, + "operator": "ge", + "actual_fraction": 0.30114084810217406, + "metric": 0.8786682930329339, + "rescued": 1375, + "harmed": 279, + "missed_rescue": 850, + "avoided_harm": 450, + "selected_both_correct": 2275, + "selected_both_wrong": 268 + }, + { + "budget": 0.31, + "threshold": -0.9167145490646362, + "operator": "ge", + "actual_fraction": 0.3110425486116094, + "metric": 0.8807490851689747, + "rescued": 1409, + "harmed": 284, + "missed_rescue": 816, + "avoided_harm": 445, + "selected_both_correct": 2369, + "selected_both_wrong": 273 + }, + { + "budget": 0.32, + "threshold": -0.9918558597564697, + "operator": "ge", + "actual_fraction": 0.3209442491210447, + "metric": 0.8816101026045777, + "rescued": 1434, + "harmed": 297, + "missed_rescue": 791, + "avoided_harm": 432, + "selected_both_correct": 2457, + "selected_both_wrong": 285 + }, + { + "budget": 0.33, + "threshold": -1.0728092193603516, + "operator": "ge", + "actual_fraction": 0.33027193800674465, + "metric": 0.8826863743990816, + "rescued": 1458, + "harmed": 306, + "missed_rescue": 767, + "avoided_harm": 423, + "selected_both_correct": 2544, + "selected_both_wrong": 295 + }, + { + "budget": 0.34, + "threshold": -1.1429637670516968, + "operator": "ge", + "actual_fraction": 0.3394561239865107, + "metric": 0.8835473918346847, + "rescued": 1481, + "harmed": 317, + "missed_rescue": 744, + "avoided_harm": 412, + "selected_both_correct": 2627, + "selected_both_wrong": 306 + }, + { + "budget": 0.35, + "threshold": -1.2190481424331665, + "operator": "ge", + "actual_fraction": 0.34914257013704525, + "metric": 0.8851259237999569, + "rescued": 1511, + "harmed": 325, + "missed_rescue": 714, + "avoided_harm": 404, + "selected_both_correct": 2710, + "selected_both_wrong": 320 + }, + { + "budget": 0.36, + "threshold": -1.3103632926940918, + "operator": "ge", + "actual_fraction": 0.35969003372318287, + "metric": 0.8868479586711631, + "rescued": 1545, + "harmed": 335, + "missed_rescue": 680, + "avoided_harm": 394, + "selected_both_correct": 2808, + "selected_both_wrong": 325 + }, + { + "budget": 0.37, + "threshold": -1.403131365776062, + "operator": "ge", + "actual_fraction": 0.37138552055679125, + "metric": 0.8880677333716007, + "rescued": 1576, + "harmed": 349, + "missed_rescue": 649, + "avoided_harm": 380, + "selected_both_correct": 2915, + "selected_both_wrong": 336 + }, + { + "budget": 0.38, + "threshold": -1.4784197807312012, + "operator": "ge", + "actual_fraction": 0.3803544521776566, + "metric": 0.8890005022601708, + "rescued": 1598, + "harmed": 358, + "missed_rescue": 627, + "avoided_harm": 371, + "selected_both_correct": 3000, + "selected_both_wrong": 345 + }, + { + "budget": 0.39, + "threshold": -1.57342529296875, + "operator": "ge", + "actual_fraction": 0.39161943029346347, + "metric": 0.8903637798665422, + "rescued": 1625, + "harmed": 366, + "missed_rescue": 600, + "avoided_harm": 363, + "selected_both_correct": 3111, + "selected_both_wrong": 356 + }, + { + "budget": 0.4, + "threshold": -1.6548807621002197, + "operator": "ge", + "actual_fraction": 0.4013776278969649, + "metric": 0.8909377914902776, + "rescued": 1640, + "harmed": 373, + "missed_rescue": 585, + "avoided_harm": 356, + "selected_both_correct": 3214, + "selected_both_wrong": 367 + }, + { + "budget": 0.41, + "threshold": -1.7329905033111572, + "operator": "ge", + "actual_fraction": 0.41135107985936714, + "metric": 0.8917988089258807, + "rescued": 1662, + "harmed": 383, + "missed_rescue": 563, + "avoided_harm": 346, + "selected_both_correct": 3317, + "selected_both_wrong": 371 + }, + { + "budget": 0.42, + "threshold": -1.8175678253173828, + "operator": "ge", + "actual_fraction": 0.42189854344550476, + "metric": 0.8924445720025831, + "rescued": 1681, + "harmed": 393, + "missed_rescue": 544, + "avoided_harm": 336, + "selected_both_correct": 3429, + "selected_both_wrong": 377 + }, + { + "budget": 0.43, + "threshold": -1.8905004262924194, + "operator": "ge", + "actual_fraction": 0.42979120327186626, + "metric": 0.8929468321733515, + "rescued": 1700, + "harmed": 405, + "missed_rescue": 525, + "avoided_harm": 324, + "selected_both_correct": 3502, + "selected_both_wrong": 383 + }, + { + "budget": 0.44, + "threshold": -1.9868656396865845, + "operator": "ge", + "actual_fraction": 0.44184544737030923, + "metric": 0.8936643467030208, + "rescued": 1720, + "harmed": 415, + "missed_rescue": 505, + "avoided_harm": 314, + "selected_both_correct": 3631, + "selected_both_wrong": 392 + }, + { + "budget": 0.45, + "threshold": -2.078063726425171, + "operator": "ge", + "actual_fraction": 0.45210590514457916, + "metric": 0.8948123699504915, + "rescued": 1741, + "harmed": 420, + "missed_rescue": 484, + "avoided_harm": 309, + "selected_both_correct": 3742, + "selected_both_wrong": 398 + }, + { + "budget": 0.46, + "threshold": -2.1543872356414795, + "operator": "ge", + "actual_fraction": 0.46164884838917986, + "metric": 0.8957451388390615, + "rescued": 1758, + "harmed": 424, + "missed_rescue": 467, + "avoided_harm": 305, + "selected_both_correct": 3850, + "selected_both_wrong": 402 + }, + { + "budget": 0.47, + "threshold": -2.22452712059021, + "operator": "ge", + "actual_fraction": 0.4699720169333429, + "metric": 0.8966779077276315, + "rescued": 1776, + "harmed": 429, + "missed_rescue": 449, + "avoided_harm": 300, + "selected_both_correct": 3942, + "selected_both_wrong": 403 + }, + { + "budget": 0.48, + "threshold": -2.2989747524261475, + "operator": "ge", + "actual_fraction": 0.4779364282126713, + "metric": 0.8975389251632345, + "rescued": 1789, + "harmed": 430, + "missed_rescue": 436, + "avoided_harm": 299, + "selected_both_correct": 4031, + "selected_both_wrong": 411 + }, + { + "budget": 0.49, + "threshold": -2.402249336242676, + "operator": "ge", + "actual_fraction": 0.49142570137045277, + "metric": 0.8983999425988376, + "rescued": 1810, + "harmed": 439, + "missed_rescue": 415, + "avoided_harm": 290, + "selected_both_correct": 4180, + "selected_both_wrong": 420 + }, + { + "budget": 0.5, + "threshold": -2.477130174636841, + "operator": "ge", + "actual_fraction": 0.4998206213675827, + "metric": 0.8990457056755399, + "rescued": 1825, + "harmed": 445, + "missed_rescue": 400, + "avoided_harm": 284, + "selected_both_correct": 4275, + "selected_both_wrong": 421 + }, + { + "budget": 0.51, + "threshold": -2.5623269081115723, + "operator": "ge", + "actual_fraction": 0.5093635646121834, + "metric": 0.8994044629403746, + "rescued": 1839, + "harmed": 454, + "missed_rescue": 386, + "avoided_harm": 275, + "selected_both_correct": 4380, + "selected_both_wrong": 426 + }, + { + "budget": 0.52, + "threshold": -2.6437463760375977, + "operator": "ge", + "actual_fraction": 0.5193370165745856, + "metric": 0.9005524861878453, + "rescued": 1862, + "harmed": 461, + "missed_rescue": 363, + "avoided_harm": 268, + "selected_both_correct": 4482, + "selected_both_wrong": 433 + }, + { + "budget": 0.53, + "threshold": -2.7279462814331055, + "operator": "ge", + "actual_fraction": 0.5303149888785248, + "metric": 0.9009829949056468, + "rescued": 1879, + "harmed": 472, + "missed_rescue": 346, + "avoided_harm": 257, + "selected_both_correct": 4606, + "selected_both_wrong": 434 + }, + { + "budget": 0.54, + "threshold": -2.832216262817383, + "operator": "ge", + "actual_fraction": 0.5427997416947693, + "metric": 0.901772260888283, + "rescued": 1897, + "harmed": 479, + "missed_rescue": 328, + "avoided_harm": 250, + "selected_both_correct": 4751, + "selected_both_wrong": 438 + }, + { + "budget": 0.55, + "threshold": -2.9212465286254883, + "operator": "ge", + "actual_fraction": 0.5536342110927747, + "metric": 0.9022745210590515, + "rescued": 1907, + "harmed": 482, + "missed_rescue": 318, + "avoided_harm": 247, + "selected_both_correct": 4883, + "selected_both_wrong": 444 + }, + { + "budget": 0.56, + "threshold": -3.0128684043884277, + "operator": "ge", + "actual_fraction": 0.5647556863026476, + "metric": 0.9024897754179522, + "rescued": 1920, + "harmed": 492, + "missed_rescue": 305, + "avoided_harm": 237, + "selected_both_correct": 5012, + "selected_both_wrong": 447 + }, + { + "budget": 0.57, + "threshold": -3.1058738231658936, + "operator": "ge", + "actual_fraction": 0.5750161440769176, + "metric": 0.9027767812298199, + "rescued": 1931, + "harmed": 499, + "missed_rescue": 294, + "avoided_harm": 230, + "selected_both_correct": 5134, + "selected_both_wrong": 450 + }, + { + "budget": 0.58, + "threshold": -3.1683619022369385, + "operator": "ge", + "actual_fraction": 0.5824065437325106, + "metric": 0.9029202841357538, + "rescued": 1941, + "harmed": 507, + "missed_rescue": 284, + "avoided_harm": 222, + "selected_both_correct": 5210, + "selected_both_wrong": 459 + }, + { + "budget": 0.59, + "threshold": -3.2430496215820312, + "operator": "ge", + "actual_fraction": 0.5905144579177728, + "metric": 0.9029202841357538, + "rescued": 1948, + "harmed": 514, + "missed_rescue": 277, + "avoided_harm": 215, + "selected_both_correct": 5305, + "selected_both_wrong": 463 + }, + { + "budget": 0.6, + "threshold": -3.325876235961914, + "operator": "ge", + "actual_fraction": 0.6008466671450097, + "metric": 0.9028485326827869, + "rescued": 1955, + "harmed": 522, + "missed_rescue": 270, + "avoided_harm": 207, + "selected_both_correct": 5432, + "selected_both_wrong": 465 + }, + { + "budget": 0.61, + "threshold": -3.407817840576172, + "operator": "ge", + "actual_fraction": 0.6106766162014781, + "metric": 0.9029920355887207, + "rescued": 1962, + "harmed": 527, + "missed_rescue": 263, + "avoided_harm": 202, + "selected_both_correct": 5552, + "selected_both_wrong": 470 + }, + { + "budget": 0.62, + "threshold": -3.487114191055298, + "operator": "ge", + "actual_fraction": 0.6207218196168472, + "metric": 0.9030637870416875, + "rescued": 1973, + "harmed": 537, + "missed_rescue": 252, + "avoided_harm": 192, + "selected_both_correct": 5671, + "selected_both_wrong": 470 + }, + { + "budget": 0.63, + "threshold": -3.5880866050720215, + "operator": "ge", + "actual_fraction": 0.6314127861089187, + "metric": 0.9034225443065222, + "rescued": 1982, + "harmed": 541, + "missed_rescue": 243, + "avoided_harm": 188, + "selected_both_correct": 5807, + "selected_both_wrong": 470 + }, + { + "budget": 0.64, + "threshold": -3.671746253967285, + "operator": "ge", + "actual_fraction": 0.6405969720886848, + "metric": 0.9037813015713568, + "rescued": 1990, + "harmed": 544, + "missed_rescue": 235, + "avoided_harm": 185, + "selected_both_correct": 5922, + "selected_both_wrong": 472 + }, + { + "budget": 0.65, + "threshold": -3.759582281112671, + "operator": "ge", + "actual_fraction": 0.6511444356748224, + "metric": 0.9039965559302576, + "rescued": 1998, + "harmed": 549, + "missed_rescue": 227, + "avoided_harm": 180, + "selected_both_correct": 6053, + "selected_both_wrong": 475 + }, + { + "budget": 0.66, + "threshold": -3.850314140319824, + "operator": "ge", + "actual_fraction": 0.6626964196024969, + "metric": 0.9044270646480591, + "rescued": 2012, + "harmed": 557, + "missed_rescue": 213, + "avoided_harm": 172, + "selected_both_correct": 6188, + "selected_both_wrong": 479 + }, + { + "budget": 0.67, + "threshold": -3.9420368671417236, + "operator": "ge", + "actual_fraction": 0.6723828657530315, + "metric": 0.904570567553993, + "rescued": 2017, + "harmed": 560, + "missed_rescue": 208, + "avoided_harm": 169, + "selected_both_correct": 6313, + "selected_both_wrong": 481 + }, + { + "budget": 0.68, + "threshold": -4.02707052230835, + "operator": "ge", + "actual_fraction": 0.6825715720743345, + "metric": 0.9051445791777284, + "rescued": 2030, + "harmed": 565, + "missed_rescue": 195, + "avoided_harm": 164, + "selected_both_correct": 6435, + "selected_both_wrong": 483 + }, + { + "budget": 0.69, + "threshold": -4.099420547485352, + "operator": "ge", + "actual_fraction": 0.690464231900696, + "metric": 0.9057185908014638, + "rescued": 2041, + "harmed": 568, + "missed_rescue": 184, + "avoided_harm": 161, + "selected_both_correct": 6531, + "selected_both_wrong": 483 + }, + { + "budget": 0.7, + "threshold": -4.196893692016602, + "operator": "ge", + "actual_fraction": 0.7016574585635359, + "metric": 0.9058620937073976, + "rescued": 2051, + "harmed": 576, + "missed_rescue": 174, + "avoided_harm": 153, + "selected_both_correct": 6665, + "selected_both_wrong": 487 + }, + { + "budget": 0.71, + "threshold": -4.279935359954834, + "operator": "ge", + "actual_fraction": 0.7111286503551697, + "metric": 0.9062926024251992, + "rescued": 2061, + "harmed": 580, + "missed_rescue": 164, + "avoided_harm": 149, + "selected_both_correct": 6778, + "selected_both_wrong": 492 + }, + { + "budget": 0.72, + "threshold": -4.365882873535156, + "operator": "ge", + "actual_fraction": 0.7211021023175719, + "metric": 0.9065796082370668, + "rescued": 2069, + "harmed": 584, + "missed_rescue": 156, + "avoided_harm": 145, + "selected_both_correct": 6904, + "selected_both_wrong": 493 + }, + { + "budget": 0.73, + "threshold": -4.45527982711792, + "operator": "ge", + "actual_fraction": 0.7304297912032719, + "metric": 0.9065796082370668, + "rescued": 2072, + "harmed": 587, + "missed_rescue": 153, + "avoided_harm": 142, + "selected_both_correct": 7025, + "selected_both_wrong": 496 + }, + { + "budget": 0.74, + "threshold": -4.553353786468506, + "operator": "ge", + "actual_fraction": 0.7406902489775418, + "metric": 0.9062208509722322, + "rescued": 2077, + "harmed": 597, + "missed_rescue": 148, + "avoided_harm": 132, + "selected_both_correct": 7152, + "selected_both_wrong": 497 + }, + { + "budget": 0.75, + "threshold": -4.638945579528809, + "operator": "ge", + "actual_fraction": 0.7494439262395064, + "metric": 0.9065796082370668, + "rescued": 2086, + "harmed": 601, + "missed_rescue": 139, + "avoided_harm": 128, + "selected_both_correct": 7259, + "selected_both_wrong": 499 + }, + { + "budget": 0.76, + "threshold": -4.725781440734863, + "operator": "ge", + "actual_fraction": 0.7579105976896032, + "metric": 0.9065796082370668, + "rescued": 2090, + "harmed": 605, + "missed_rescue": 135, + "avoided_harm": 124, + "selected_both_correct": 7368, + "selected_both_wrong": 500 + }, + { + "budget": 0.77, + "threshold": -4.809637069702148, + "operator": "ge", + "actual_fraction": 0.7660902633278324, + "metric": 0.9065078567840998, + "rescued": 2093, + "harmed": 609, + "missed_rescue": 132, + "avoided_harm": 120, + "selected_both_correct": 7473, + "selected_both_wrong": 502 + }, + { + "budget": 0.78, + "threshold": -4.926034927368164, + "operator": "ge", + "actual_fraction": 0.7778575016144077, + "metric": 0.9066513596900337, + "rescued": 2103, + "harmed": 617, + "missed_rescue": 122, + "avoided_harm": 112, + "selected_both_correct": 7613, + "selected_both_wrong": 508 + }, + { + "budget": 0.79, + "threshold": -5.015888690948486, + "operator": "ge", + "actual_fraction": 0.7870416875941738, + "metric": 0.9069383655019014, + "rescued": 2110, + "harmed": 620, + "missed_rescue": 115, + "avoided_harm": 109, + "selected_both_correct": 7729, + "selected_both_wrong": 510 + }, + { + "budget": 0.8, + "threshold": -5.120303630828857, + "operator": "ge", + "actual_fraction": 0.7977326540862453, + "metric": 0.9068666140489345, + "rescued": 2116, + "harmed": 627, + "missed_rescue": 109, + "avoided_harm": 102, + "selected_both_correct": 7861, + "selected_both_wrong": 514 + }, + { + "budget": 0.81, + "threshold": -5.253293037414551, + "operator": "ge", + "actual_fraction": 0.8112936786969936, + "metric": 0.906436105331133, + "rescued": 2122, + "harmed": 639, + "missed_rescue": 103, + "avoided_harm": 90, + "selected_both_correct": 8032, + "selected_both_wrong": 514 + }, + { + "budget": 0.82, + "threshold": -5.367966175079346, + "operator": "ge", + "actual_fraction": 0.8213388821123627, + "metric": 0.906436105331133, + "rescued": 2129, + "harmed": 646, + "missed_rescue": 96, + "avoided_harm": 83, + "selected_both_correct": 8157, + "selected_both_wrong": 515 + }, + { + "budget": 0.83, + "threshold": -5.470327854156494, + "operator": "ge", + "actual_fraction": 0.830379565186195, + "metric": 0.906364353878166, + "rescued": 2133, + "harmed": 651, + "missed_rescue": 92, + "avoided_harm": 78, + "selected_both_correct": 8272, + "selected_both_wrong": 517 + }, + { + "budget": 0.84, + "threshold": -5.563735008239746, + "operator": "ge", + "actual_fraction": 0.8376264619358542, + "metric": 0.906436105331133, + "rescued": 2137, + "harmed": 654, + "missed_rescue": 88, + "avoided_harm": 75, + "selected_both_correct": 8363, + "selected_both_wrong": 520 + }, + { + "budget": 0.85, + "threshold": -5.675332069396973, + "operator": "ge", + "actual_fraction": 0.8465953935567195, + "metric": 0.906436105331133, + "rescued": 2140, + "harmed": 657, + "missed_rescue": 85, + "avoided_harm": 72, + "selected_both_correct": 8480, + "selected_both_wrong": 522 + }, + { + "budget": 0.86, + "threshold": -5.787599086761475, + "operator": "ge", + "actual_fraction": 0.8551338164597833, + "metric": 0.9070101169548683, + "rescued": 2152, + "harmed": 661, + "missed_rescue": 73, + "avoided_harm": 68, + "selected_both_correct": 8583, + "selected_both_wrong": 522 + }, + { + "budget": 0.87, + "threshold": -5.91396951675415, + "operator": "ge", + "actual_fraction": 0.8662552916696563, + "metric": 0.9071536198608022, + "rescued": 2158, + "harmed": 665, + "missed_rescue": 67, + "avoided_harm": 64, + "selected_both_correct": 8727, + "selected_both_wrong": 523 + }, + { + "budget": 0.88, + "threshold": -6.055293083190918, + "operator": "ge", + "actual_fraction": 0.8775202697854632, + "metric": 0.9073688742197029, + "rescued": 2166, + "harmed": 670, + "missed_rescue": 59, + "avoided_harm": 59, + "selected_both_correct": 8867, + "selected_both_wrong": 527 + }, + { + "budget": 0.89, + "threshold": -6.180037975311279, + "operator": "ge", + "actual_fraction": 0.8864892014063285, + "metric": 0.9075841285786037, + "rescued": 2172, + "harmed": 673, + "missed_rescue": 53, + "avoided_harm": 56, + "selected_both_correct": 8980, + "selected_both_wrong": 530 + }, + { + "budget": 0.9, + "threshold": -6.315410614013672, + "operator": "ge", + "actual_fraction": 0.8963909019157638, + "metric": 0.9077276314845376, + "rescued": 2179, + "harmed": 678, + "missed_rescue": 46, + "avoided_harm": 51, + "selected_both_correct": 9102, + "selected_both_wrong": 534 + }, + { + "budget": 0.91, + "threshold": -6.488847732543945, + "operator": "ge", + "actual_fraction": 0.9072253713137691, + "metric": 0.9080863887493722, + "rescued": 2185, + "harmed": 679, + "missed_rescue": 40, + "avoided_harm": 50, + "selected_both_correct": 9244, + "selected_both_wrong": 536 + }, + { + "budget": 0.92, + "threshold": -6.6685333251953125, + "operator": "ge", + "actual_fraction": 0.9186338523355098, + "metric": 0.9084451460142068, + "rescued": 2195, + "harmed": 684, + "missed_rescue": 30, + "avoided_harm": 45, + "selected_both_correct": 9384, + "selected_both_wrong": 540 + }, + { + "budget": 0.93, + "threshold": -6.833990097045898, + "operator": "ge", + "actual_fraction": 0.927746286862309, + "metric": 0.908229891655306, + "rescued": 2198, + "harmed": 690, + "missed_rescue": 27, + "avoided_harm": 39, + "selected_both_correct": 9497, + "selected_both_wrong": 545 + }, + { + "budget": 0.94, + "threshold": -7.018073558807373, + "operator": "ge", + "actual_fraction": 0.9376479873717443, + "metric": 0.9081581402023391, + "rescued": 2203, + "harmed": 696, + "missed_rescue": 22, + "avoided_harm": 33, + "selected_both_correct": 9622, + "selected_both_wrong": 547 + }, + { + "budget": 0.95, + "threshold": -7.223591327667236, + "operator": "ge", + "actual_fraction": 0.9474779364282127, + "metric": 0.9080146372964053, + "rescued": 2207, + "harmed": 702, + "missed_rescue": 18, + "avoided_harm": 27, + "selected_both_correct": 9748, + "selected_both_wrong": 548 + }, + { + "budget": 0.96, + "threshold": -7.463536262512207, + "operator": "ge", + "actual_fraction": 0.9585994116380857, + "metric": 0.9078711343904714, + "rescued": 2210, + "harmed": 707, + "missed_rescue": 15, + "avoided_harm": 22, + "selected_both_correct": 9895, + "selected_both_wrong": 548 + }, + { + "budget": 0.97, + "threshold": -7.692946434020996, + "operator": "ge", + "actual_fraction": 0.9660615627466457, + "metric": 0.9075841285786037, + "rescued": 2210, + "harmed": 711, + "missed_rescue": 15, + "avoided_harm": 18, + "selected_both_correct": 9992, + "selected_both_wrong": 551 + }, + { + "budget": 0.98, + "threshold": -8.048358917236328, + "operator": "ge", + "actual_fraction": 0.9768960321446509, + "metric": 0.9077993829375045, + "rescued": 2218, + "harmed": 716, + "missed_rescue": 7, + "avoided_harm": 13, + "selected_both_correct": 10128, + "selected_both_wrong": 553 + }, + { + "budget": 0.99, + "threshold": -8.652192115783691, + "operator": "ge", + "actual_fraction": 0.9901700509435316, + "metric": 0.9073688742197029, + "rescued": 2221, + "harmed": 725, + "missed_rescue": 4, + "avoided_harm": 4, + "selected_both_correct": 10298, + "selected_both_wrong": 556 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9073688742197029, + "rescued": 2225, + "harmed": 729, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 10421, + "selected_both_wrong": 562 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 5190, + "rescuable": 1087, + "harmful": 383, + "both_wrong": 322, + "unit": "sample", + "total": 6982 + }, + "test": { + "both_correct": 10421, + "rescuable": 2225, + "harmful": 729, + "both_wrong": 562, + "unit": "sample", + "total": 13937 + } + }, + "small": { + "validation": 0.7981953594958464, + "test": 0.8000287005811868 + }, + "large": { + "validation": 0.8990260670295045, + "test": 0.9073688742197029 + }, + "learned_minus_confidence_test": -0.0012915261534046651, + "learned_minus_small_test": 0.10640740474994614, + "learned_minus_large_test": -0.0009327688885699681 + }, + { + "key": [ + "nlvr2", + "nlvr2", + "Qwen3.8" + ], + "expert_display": "BEiT3", + "run": "shared_12_nlvr2_Qwen3.8", + "validation_rows": 6982, + "test_rows": 13937, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/nlvr2/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 86, + "validation_area_0_25": 0.8312804354053279, + "validation_point": { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8587797192781438, + "metric": 0.9057576625608708, + "rescued": 1134, + "harmed": 383, + "missed_rescue": 20, + "avoided_harm": 25, + "selected_both_correct": 4230, + "selected_both_wrong": 249 + }, + "test_point": { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8580038745784602, + "metric": 0.9024180239649853, + "rescued": 2213, + "harmed": 786, + "missed_rescue": 34, + "avoided_harm": 59, + "selected_both_correct": 8437, + "selected_both_wrong": 522 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7981953594958464, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1154, + "avoided_harm": 408, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5245773792266846, + "operator": "gt", + "actual_fraction": 0.0097393297049556, + "metric": 0.8004869664852478, + "rescued": 26, + "harmed": 10, + "missed_rescue": 1128, + "avoided_harm": 398, + "selected_both_correct": 30, + "selected_both_wrong": 2 + }, + { + "budget": 0.02, + "threshold": -0.5456489324569702, + "operator": "gt", + "actual_fraction": 0.019765110283586365, + "metric": 0.8033514752219995, + "rescued": 51, + "harmed": 15, + "missed_rescue": 1103, + "avoided_harm": 393, + "selected_both_correct": 60, + "selected_both_wrong": 12 + }, + { + "budget": 0.03, + "threshold": -0.5652987360954285, + "operator": "ge", + "actual_fraction": 0.029934116299054712, + "metric": 0.8067888857061014, + "rescued": 83, + "harmed": 23, + "missed_rescue": 1071, + "avoided_harm": 385, + "selected_both_correct": 84, + "selected_both_wrong": 19 + }, + { + "budget": 0.04, + "threshold": -0.5851011276245117, + "operator": "ge", + "actual_fraction": 0.039959896877685475, + "metric": 0.8087940418218276, + "rescued": 105, + "harmed": 31, + "missed_rescue": 1049, + "avoided_harm": 377, + "selected_both_correct": 117, + "selected_both_wrong": 26 + }, + { + "budget": 0.05, + "threshold": -0.6055658459663391, + "operator": "ge", + "actual_fraction": 0.04998567745631624, + "metric": 0.8110856488112289, + "rescued": 125, + "harmed": 35, + "missed_rescue": 1029, + "avoided_harm": 373, + "selected_both_correct": 157, + "selected_both_wrong": 32 + }, + { + "budget": 0.06, + "threshold": -0.6293190717697144, + "operator": "ge", + "actual_fraction": 0.059868232598109423, + "metric": 0.8140933829848181, + "rescued": 152, + "harmed": 41, + "missed_rescue": 1002, + "avoided_harm": 367, + "selected_both_correct": 187, + "selected_both_wrong": 38 + }, + { + "budget": 0.07, + "threshold": -0.6477982401847839, + "operator": "ge", + "actual_fraction": 0.06989401317674018, + "metric": 0.8179604697794328, + "rescued": 183, + "harmed": 45, + "missed_rescue": 971, + "avoided_harm": 363, + "selected_both_correct": 214, + "selected_both_wrong": 46 + }, + { + "budget": 0.08, + "threshold": -0.6693127155303955, + "operator": "ge", + "actual_fraction": 0.07991979375537095, + "metric": 0.8212546548266972, + "rescued": 211, + "harmed": 50, + "missed_rescue": 943, + "avoided_harm": 358, + "selected_both_correct": 246, + "selected_both_wrong": 51 + }, + { + "budget": 0.09, + "threshold": -0.6888871788978577, + "operator": "gt", + "actual_fraction": 0.08951589802348897, + "metric": 0.8239759381266113, + "rescued": 238, + "harmed": 58, + "missed_rescue": 916, + "avoided_harm": 350, + "selected_both_correct": 275, + "selected_both_wrong": 54 + }, + { + "budget": 0.1, + "threshold": -0.7061904072761536, + "operator": "gt", + "actual_fraction": 0.09925522772844457, + "metric": 0.8252649670581496, + "rescued": 255, + "harmed": 66, + "missed_rescue": 899, + "avoided_harm": 342, + "selected_both_correct": 311, + "selected_both_wrong": 61 + }, + { + "budget": 0.11, + "threshold": -0.7233095169067383, + "operator": "ge", + "actual_fraction": 0.10999713549126325, + "metric": 0.8289888284159267, + "rescued": 284, + "harmed": 69, + "missed_rescue": 870, + "avoided_harm": 339, + "selected_both_correct": 350, + "selected_both_wrong": 65 + }, + { + "budget": 0.12, + "threshold": -0.7424216866493225, + "operator": "ge", + "actual_fraction": 0.11987969063305644, + "metric": 0.8319965625895159, + "rescued": 314, + "harmed": 78, + "missed_rescue": 840, + "avoided_harm": 330, + "selected_both_correct": 377, + "selected_both_wrong": 68 + }, + { + "budget": 0.13, + "threshold": -0.7627776861190796, + "operator": "gt", + "actual_fraction": 0.1297622457748496, + "metric": 0.8344313950157548, + "rescued": 338, + "harmed": 85, + "missed_rescue": 816, + "avoided_harm": 323, + "selected_both_correct": 411, + "selected_both_wrong": 72 + }, + { + "budget": 0.14, + "threshold": -0.7786493301391602, + "operator": "gt", + "actual_fraction": 0.1396448009166428, + "metric": 0.8371526783156689, + "rescued": 366, + "harmed": 94, + "missed_rescue": 788, + "avoided_harm": 314, + "selected_both_correct": 440, + "selected_both_wrong": 75 + }, + { + "budget": 0.15, + "threshold": -0.7950222492218018, + "operator": "ge", + "actual_fraction": 0.14995703236894872, + "metric": 0.8400171870524205, + "rescued": 390, + "harmed": 98, + "missed_rescue": 764, + "avoided_harm": 310, + "selected_both_correct": 477, + "selected_both_wrong": 82 + }, + { + "budget": 0.16, + "threshold": -0.8140527606010437, + "operator": "ge", + "actual_fraction": 0.1599828129475795, + "metric": 0.8424520194786594, + "rescued": 412, + "harmed": 103, + "missed_rescue": 742, + "avoided_harm": 305, + "selected_both_correct": 514, + "selected_both_wrong": 88 + }, + { + "budget": 0.17, + "threshold": -0.8327813148498535, + "operator": "ge", + "actual_fraction": 0.16986536808937266, + "metric": 0.844313950157548, + "rescued": 429, + "harmed": 107, + "missed_rescue": 725, + "avoided_harm": 301, + "selected_both_correct": 556, + "selected_both_wrong": 94 + }, + { + "budget": 0.18, + "threshold": -0.8464508652687073, + "operator": "gt", + "actual_fraction": 0.17960469779432828, + "metric": 0.8463191062732741, + "rescued": 446, + "harmed": 110, + "missed_rescue": 708, + "avoided_harm": 298, + "selected_both_correct": 600, + "selected_both_wrong": 98 + }, + { + "budget": 0.19, + "threshold": -0.858244001865387, + "operator": "ge", + "actual_fraction": 0.1899169292466342, + "metric": 0.8490403895731882, + "rescued": 470, + "harmed": 115, + "missed_rescue": 684, + "avoided_harm": 293, + "selected_both_correct": 637, + "selected_both_wrong": 104 + }, + { + "budget": 0.2, + "threshold": -0.8701565861701965, + "operator": "ge", + "actual_fraction": 0.19994270982526496, + "metric": 0.8509023202520768, + "rescued": 488, + "harmed": 120, + "missed_rescue": 666, + "avoided_harm": 288, + "selected_both_correct": 680, + "selected_both_wrong": 108 + }, + { + "budget": 0.21, + "threshold": -0.8840392827987671, + "operator": "gt", + "actual_fraction": 0.2093955886565454, + "metric": 0.852191349183615, + "rescued": 505, + "harmed": 128, + "missed_rescue": 649, + "avoided_harm": 280, + "selected_both_correct": 718, + "selected_both_wrong": 111 + }, + { + "budget": 0.22, + "threshold": -0.8933094143867493, + "operator": "ge", + "actual_fraction": 0.2199942709825265, + "metric": 0.8533371526783157, + "rescued": 522, + "harmed": 137, + "missed_rescue": 632, + "avoided_harm": 271, + "selected_both_correct": 762, + "selected_both_wrong": 115 + }, + { + "budget": 0.23, + "threshold": -0.9046505093574524, + "operator": "gt", + "actual_fraction": 0.2297336006874821, + "metric": 0.855628759667717, + "rescued": 543, + "harmed": 142, + "missed_rescue": 611, + "avoided_harm": 266, + "selected_both_correct": 799, + "selected_both_wrong": 120 + }, + { + "budget": 0.24, + "threshold": -0.9136765599250793, + "operator": "ge", + "actual_fraction": 0.23990260670295044, + "metric": 0.8583500429676311, + "rescued": 566, + "harmed": 146, + "missed_rescue": 588, + "avoided_harm": 262, + "selected_both_correct": 836, + "selected_both_wrong": 127 + }, + { + "budget": 0.25, + "threshold": -0.9230391979217529, + "operator": "gt", + "actual_fraction": 0.24906903466055572, + "metric": 0.860068748209682, + "rescued": 585, + "harmed": 153, + "missed_rescue": 569, + "avoided_harm": 255, + "selected_both_correct": 871, + "selected_both_wrong": 130 + }, + { + "budget": 0.26, + "threshold": -0.9314625263214111, + "operator": "gt", + "actual_fraction": 0.2595244915496992, + "metric": 0.8632197078201088, + "rescued": 610, + "harmed": 156, + "missed_rescue": 544, + "avoided_harm": 252, + "selected_both_correct": 914, + "selected_both_wrong": 132 + }, + { + "budget": 0.27, + "threshold": -0.9376689791679382, + "operator": "ge", + "actual_fraction": 0.26997994843884277, + "metric": 0.8656545402463478, + "rescued": 630, + "harmed": 159, + "missed_rescue": 524, + "avoided_harm": 249, + "selected_both_correct": 962, + "selected_both_wrong": 134 + }, + { + "budget": 0.28, + "threshold": -0.9449946880340576, + "operator": "gt", + "actual_fraction": 0.2794328272701232, + "metric": 0.8673732454883988, + "rescued": 645, + "harmed": 162, + "missed_rescue": 509, + "avoided_harm": 246, + "selected_both_correct": 1005, + "selected_both_wrong": 139 + }, + { + "budget": 0.29, + "threshold": -0.9511421918869019, + "operator": "gt", + "actual_fraction": 0.2896018332855915, + "metric": 0.8689487252936121, + "rescued": 664, + "harmed": 170, + "missed_rescue": 490, + "avoided_harm": 238, + "selected_both_correct": 1043, + "selected_both_wrong": 145 + }, + { + "budget": 0.3, + "threshold": -0.9559813141822815, + "operator": "gt", + "actual_fraction": 0.29977083930105985, + "metric": 0.870380979661988, + "rescued": 681, + "harmed": 177, + "missed_rescue": 473, + "avoided_harm": 231, + "selected_both_correct": 1083, + "selected_both_wrong": 152 + }, + { + "budget": 0.31, + "threshold": -0.9597620368003845, + "operator": "ge", + "actual_fraction": 0.30993984531652824, + "metric": 0.8716700085935262, + "rescued": 697, + "harmed": 184, + "missed_rescue": 457, + "avoided_harm": 224, + "selected_both_correct": 1129, + "selected_both_wrong": 154 + }, + { + "budget": 0.32, + "threshold": -0.9643214344978333, + "operator": "gt", + "actual_fraction": 0.3196791750214838, + "metric": 0.8743912918934402, + "rescued": 719, + "harmed": 187, + "missed_rescue": 435, + "avoided_harm": 221, + "selected_both_correct": 1171, + "selected_both_wrong": 155 + }, + { + "budget": 0.33, + "threshold": -0.9686195850372314, + "operator": "gt", + "actual_fraction": 0.3297049556001146, + "metric": 0.8746777427671154, + "rescued": 731, + "harmed": 197, + "missed_rescue": 423, + "avoided_harm": 211, + "selected_both_correct": 1215, + "selected_both_wrong": 159 + }, + { + "budget": 0.34, + "threshold": -0.9719924926757812, + "operator": "gt", + "actual_fraction": 0.339157834431395, + "metric": 0.8771125751933544, + "rescued": 751, + "harmed": 200, + "missed_rescue": 403, + "avoided_harm": 208, + "selected_both_correct": 1256, + "selected_both_wrong": 161 + }, + { + "budget": 0.35, + "threshold": -0.974821150302887, + "operator": "ge", + "actual_fraction": 0.3498997421942137, + "metric": 0.8786880549985677, + "rescued": 767, + "harmed": 205, + "missed_rescue": 387, + "avoided_harm": 203, + "selected_both_correct": 1306, + "selected_both_wrong": 165 + }, + { + "budget": 0.36, + "threshold": -0.9777138829231262, + "operator": "gt", + "actual_fraction": 0.3587797192781438, + "metric": 0.8814093382984818, + "rescued": 786, + "harmed": 205, + "missed_rescue": 368, + "avoided_harm": 203, + "selected_both_correct": 1345, + "selected_both_wrong": 169 + }, + { + "budget": 0.37, + "threshold": -0.9802808165550232, + "operator": "gt", + "actual_fraction": 0.3685190489830994, + "metric": 0.881695789172157, + "rescued": 797, + "harmed": 214, + "missed_rescue": 357, + "avoided_harm": 194, + "selected_both_correct": 1390, + "selected_both_wrong": 172 + }, + { + "budget": 0.38, + "threshold": -0.982557475566864, + "operator": "gt", + "actual_fraction": 0.3795474076195932, + "metric": 0.8839873961615583, + "rescued": 817, + "harmed": 218, + "missed_rescue": 337, + "avoided_harm": 190, + "selected_both_correct": 1439, + "selected_both_wrong": 176 + }, + { + "budget": 0.39, + "threshold": -0.9843363761901855, + "operator": "gt", + "actual_fraction": 0.3892867373245488, + "metric": 0.8847035233457462, + "rescued": 827, + "harmed": 223, + "missed_rescue": 327, + "avoided_harm": 185, + "selected_both_correct": 1487, + "selected_both_wrong": 181 + }, + { + "budget": 0.4, + "threshold": -0.9863631129264832, + "operator": "gt", + "actual_fraction": 0.3993125179031796, + "metric": 0.8864222285877972, + "rescued": 843, + "harmed": 227, + "missed_rescue": 311, + "avoided_harm": 181, + "selected_both_correct": 1536, + "selected_both_wrong": 182 + }, + { + "budget": 0.41, + "threshold": -0.9879462122917175, + "operator": "gt", + "actual_fraction": 0.40962474935548554, + "metric": 0.8879977083930106, + "rescued": 857, + "harmed": 230, + "missed_rescue": 297, + "avoided_harm": 178, + "selected_both_correct": 1586, + "selected_both_wrong": 187 + }, + { + "budget": 0.42, + "threshold": -0.9890130758285522, + "operator": "gt", + "actual_fraction": 0.41850472643941566, + "metric": 0.8897164136350616, + "rescued": 872, + "harmed": 233, + "missed_rescue": 282, + "avoided_harm": 175, + "selected_both_correct": 1628, + "selected_both_wrong": 189 + }, + { + "budget": 0.43, + "threshold": -0.9899863600730896, + "operator": "gt", + "actual_fraction": 0.4270982526496706, + "metric": 0.890575766256087, + "rescued": 882, + "harmed": 237, + "missed_rescue": 272, + "avoided_harm": 171, + "selected_both_correct": 1674, + "selected_both_wrong": 189 + }, + { + "budget": 0.44, + "threshold": -0.9911527037620544, + "operator": "gt", + "actual_fraction": 0.4395588656545402, + "metric": 0.8928673732454884, + "rescued": 903, + "harmed": 242, + "missed_rescue": 251, + "avoided_harm": 166, + "selected_both_correct": 1728, + "selected_both_wrong": 196 + }, + { + "budget": 0.45, + "threshold": -0.9919379353523254, + "operator": "gt", + "actual_fraction": 0.4497278716700086, + "metric": 0.8937267258665139, + "rescued": 913, + "harmed": 246, + "missed_rescue": 241, + "avoided_harm": 162, + "selected_both_correct": 1784, + "selected_both_wrong": 197 + }, + { + "budget": 0.46, + "threshold": -0.9926542043685913, + "operator": "gt", + "actual_fraction": 0.4586078487539387, + "metric": 0.8953022056717272, + "rescued": 925, + "harmed": 247, + "missed_rescue": 229, + "avoided_harm": 161, + "selected_both_correct": 1832, + "selected_both_wrong": 198 + }, + { + "budget": 0.47, + "threshold": -0.9933071732521057, + "operator": "gt", + "actual_fraction": 0.46863362933256947, + "metric": 0.8961615582927528, + "rescued": 935, + "harmed": 251, + "missed_rescue": 219, + "avoided_harm": 157, + "selected_both_correct": 1886, + "selected_both_wrong": 200 + }, + { + "budget": 0.48, + "threshold": -0.9939024448394775, + "operator": "gt", + "actual_fraction": 0.4798052134059009, + "metric": 0.8960183328559153, + "rescued": 941, + "harmed": 258, + "missed_rescue": 213, + "avoided_harm": 150, + "selected_both_correct": 1948, + "selected_both_wrong": 203 + }, + { + "budget": 0.49, + "threshold": -0.9944451451301575, + "operator": "gt", + "actual_fraction": 0.4891148668003437, + "metric": 0.8970209109137783, + "rescued": 953, + "harmed": 263, + "missed_rescue": 201, + "avoided_harm": 145, + "selected_both_correct": 1992, + "selected_both_wrong": 207 + }, + { + "budget": 0.5, + "threshold": -0.9949396848678589, + "operator": "gt", + "actual_fraction": 0.4992838728158121, + "metric": 0.897450587224291, + "rescued": 960, + "harmed": 267, + "missed_rescue": 194, + "avoided_harm": 141, + "selected_both_correct": 2048, + "selected_both_wrong": 211 + }, + { + "budget": 0.51, + "threshold": -0.9952448010444641, + "operator": "gt", + "actual_fraction": 0.5065883700945288, + "metric": 0.8978802635348038, + "rescued": 965, + "harmed": 269, + "missed_rescue": 189, + "avoided_harm": 139, + "selected_both_correct": 2091, + "selected_both_wrong": 212 + }, + { + "budget": 0.52, + "threshold": -0.9956685304641724, + "operator": "gt", + "actual_fraction": 0.5179031796046978, + "metric": 0.8977370380979662, + "rescued": 972, + "harmed": 277, + "missed_rescue": 182, + "avoided_harm": 131, + "selected_both_correct": 2154, + "selected_both_wrong": 213 + }, + { + "budget": 0.53, + "threshold": -0.9961755275726318, + "operator": "gt", + "actual_fraction": 0.528788312804354, + "metric": 0.8980234889716414, + "rescued": 980, + "harmed": 283, + "missed_rescue": 174, + "avoided_harm": 125, + "selected_both_correct": 2214, + "selected_both_wrong": 215 + }, + { + "budget": 0.54, + "threshold": -0.9965165853500366, + "operator": "gt", + "actual_fraction": 0.5380979661987969, + "metric": 0.8988828415926668, + "rescued": 989, + "harmed": 286, + "missed_rescue": 165, + "avoided_harm": 122, + "selected_both_correct": 2267, + "selected_both_wrong": 215 + }, + { + "budget": 0.55, + "threshold": -0.9968273043632507, + "operator": "gt", + "actual_fraction": 0.5489830993984531, + "metric": 0.898166714408479, + "rescued": 993, + "harmed": 295, + "missed_rescue": 161, + "avoided_harm": 113, + "selected_both_correct": 2330, + "selected_both_wrong": 215 + }, + { + "budget": 0.56, + "threshold": -0.9971103668212891, + "operator": "gt", + "actual_fraction": 0.5592953308507591, + "metric": 0.8987396161558293, + "rescued": 1001, + "harmed": 299, + "missed_rescue": 153, + "avoided_harm": 109, + "selected_both_correct": 2387, + "selected_both_wrong": 218 + }, + { + "budget": 0.57, + "threshold": -0.997285008430481, + "operator": "gt", + "actual_fraction": 0.5690346605557147, + "metric": 0.899169292466342, + "rescued": 1008, + "harmed": 303, + "missed_rescue": 146, + "avoided_harm": 105, + "selected_both_correct": 2440, + "selected_both_wrong": 222 + }, + { + "budget": 0.58, + "threshold": -0.9974491000175476, + "operator": "gt", + "actual_fraction": 0.5783443139501575, + "metric": 0.899169292466342, + "rescued": 1012, + "harmed": 307, + "missed_rescue": 142, + "avoided_harm": 101, + "selected_both_correct": 2492, + "selected_both_wrong": 227 + }, + { + "budget": 0.59, + "threshold": -0.9976769089698792, + "operator": "gt", + "actual_fraction": 0.5893726725866514, + "metric": 0.899169292466342, + "rescued": 1017, + "harmed": 312, + "missed_rescue": 137, + "avoided_harm": 96, + "selected_both_correct": 2558, + "selected_both_wrong": 228 + }, + { + "budget": 0.6, + "threshold": -0.9978172779083252, + "operator": "gt", + "actual_fraction": 0.5976797479232312, + "metric": 0.8998854196505299, + "rescued": 1024, + "harmed": 314, + "missed_rescue": 130, + "avoided_harm": 94, + "selected_both_correct": 2606, + "selected_both_wrong": 229 + }, + { + "budget": 0.61, + "threshold": -0.9980121850967407, + "operator": "gt", + "actual_fraction": 0.6084216556860499, + "metric": 0.9007447722715555, + "rescued": 1033, + "harmed": 317, + "missed_rescue": 121, + "avoided_harm": 91, + "selected_both_correct": 2669, + "selected_both_wrong": 229 + }, + { + "budget": 0.62, + "threshold": -0.9981897473335266, + "operator": "gt", + "actual_fraction": 0.6187338871383558, + "metric": 0.9004583213978803, + "rescued": 1036, + "harmed": 322, + "missed_rescue": 118, + "avoided_harm": 86, + "selected_both_correct": 2733, + "selected_both_wrong": 229 + }, + { + "budget": 0.63, + "threshold": -0.9982993006706238, + "operator": "gt", + "actual_fraction": 0.6280435405327986, + "metric": 0.9014608994557434, + "rescued": 1047, + "harmed": 326, + "missed_rescue": 107, + "avoided_harm": 82, + "selected_both_correct": 2783, + "selected_both_wrong": 229 + }, + { + "budget": 0.64, + "threshold": -0.9984021782875061, + "operator": "gt", + "actual_fraction": 0.6360641649957033, + "metric": 0.9018905757662561, + "rescued": 1054, + "harmed": 330, + "missed_rescue": 100, + "avoided_harm": 78, + "selected_both_correct": 2828, + "selected_both_wrong": 229 + }, + { + "budget": 0.65, + "threshold": -0.9985449314117432, + "operator": "gt", + "actual_fraction": 0.6465196218848468, + "metric": 0.9021770266399313, + "rescued": 1057, + "harmed": 331, + "missed_rescue": 97, + "avoided_harm": 77, + "selected_both_correct": 2897, + "selected_both_wrong": 229 + }, + { + "budget": 0.66, + "threshold": -0.9986749291419983, + "operator": "gt", + "actual_fraction": 0.6586937840160413, + "metric": 0.9024634775136064, + "rescued": 1065, + "harmed": 337, + "missed_rescue": 89, + "avoided_harm": 71, + "selected_both_correct": 2965, + "selected_both_wrong": 232 + }, + { + "budget": 0.67, + "threshold": -0.9987550973892212, + "operator": "gt", + "actual_fraction": 0.6672873102262962, + "metric": 0.902606702950444, + "rescued": 1069, + "harmed": 340, + "missed_rescue": 85, + "avoided_harm": 68, + "selected_both_correct": 3017, + "selected_both_wrong": 233 + }, + { + "budget": 0.68, + "threshold": -0.9988664388656616, + "operator": "gt", + "actual_fraction": 0.6793182469206531, + "metric": 0.9034660555714695, + "rescued": 1079, + "harmed": 344, + "missed_rescue": 75, + "avoided_harm": 64, + "selected_both_correct": 3086, + "selected_both_wrong": 234 + }, + { + "budget": 0.69, + "threshold": -0.9989677667617798, + "operator": "gt", + "actual_fraction": 0.6894872529361215, + "metric": 0.9038957318819822, + "rescued": 1085, + "harmed": 347, + "missed_rescue": 69, + "avoided_harm": 61, + "selected_both_correct": 3147, + "selected_both_wrong": 235 + }, + { + "budget": 0.7, + "threshold": -0.9990302324295044, + "operator": "gt", + "actual_fraction": 0.6969349756516757, + "metric": 0.9033228301346319, + "rescued": 1085, + "harmed": 351, + "missed_rescue": 69, + "avoided_harm": 57, + "selected_both_correct": 3195, + "selected_both_wrong": 235 + }, + { + "budget": 0.71, + "threshold": -0.9991169571876526, + "operator": "gt", + "actual_fraction": 0.7098252649670581, + "metric": 0.9037525064451447, + "rescued": 1091, + "harmed": 354, + "missed_rescue": 63, + "avoided_harm": 54, + "selected_both_correct": 3274, + "selected_both_wrong": 237 + }, + { + "budget": 0.72, + "threshold": -0.9991704225540161, + "operator": "gt", + "actual_fraction": 0.7189916929246635, + "metric": 0.9040389573188198, + "rescued": 1094, + "harmed": 355, + "missed_rescue": 60, + "avoided_harm": 53, + "selected_both_correct": 3334, + "selected_both_wrong": 237 + }, + { + "budget": 0.73, + "threshold": -0.9992445707321167, + "operator": "gt", + "actual_fraction": 0.7295903752506445, + "metric": 0.9038957318819822, + "rescued": 1098, + "harmed": 360, + "missed_rescue": 56, + "avoided_harm": 48, + "selected_both_correct": 3399, + "selected_both_wrong": 237 + }, + { + "budget": 0.74, + "threshold": -0.9992903470993042, + "operator": "gt", + "actual_fraction": 0.7384703523345746, + "metric": 0.9040389573188198, + "rescued": 1103, + "harmed": 364, + "missed_rescue": 51, + "avoided_harm": 44, + "selected_both_correct": 3450, + "selected_both_wrong": 239 + }, + { + "budget": 0.75, + "threshold": -0.9993333220481873, + "operator": "gt", + "actual_fraction": 0.7484961329132054, + "metric": 0.9044686336293326, + "rescued": 1108, + "harmed": 366, + "missed_rescue": 46, + "avoided_harm": 42, + "selected_both_correct": 3513, + "selected_both_wrong": 239 + }, + { + "budget": 0.76, + "threshold": -0.999373733997345, + "operator": "gt", + "actual_fraction": 0.7553709538814093, + "metric": 0.9041821827556574, + "rescued": 1109, + "harmed": 369, + "missed_rescue": 45, + "avoided_harm": 39, + "selected_both_correct": 3557, + "selected_both_wrong": 239 + }, + { + "budget": 0.77, + "threshold": -0.9994296431541443, + "operator": "gt", + "actual_fraction": 0.7694070466914924, + "metric": 0.9047550845030078, + "rescued": 1115, + "harmed": 371, + "missed_rescue": 39, + "avoided_harm": 37, + "selected_both_correct": 3645, + "selected_both_wrong": 241 + }, + { + "budget": 0.78, + "threshold": -0.999464213848114, + "operator": "gt", + "actual_fraction": 0.7782870237754225, + "metric": 0.9050415353766829, + "rescued": 1117, + "harmed": 371, + "missed_rescue": 37, + "avoided_harm": 37, + "selected_both_correct": 3702, + "selected_both_wrong": 244 + }, + { + "budget": 0.79, + "threshold": -0.9994966983795166, + "operator": "gt", + "actual_fraction": 0.7885992552277284, + "metric": 0.9051847608135205, + "rescued": 1119, + "harmed": 372, + "missed_rescue": 35, + "avoided_harm": 36, + "selected_both_correct": 3770, + "selected_both_wrong": 245 + }, + { + "budget": 0.8, + "threshold": -0.9995417594909668, + "operator": "gt", + "actual_fraction": 0.7977656831853337, + "metric": 0.9048983099398453, + "rescued": 1120, + "harmed": 375, + "missed_rescue": 34, + "avoided_harm": 33, + "selected_both_correct": 3829, + "selected_both_wrong": 246 + }, + { + "budget": 0.81, + "threshold": -0.9995827078819275, + "operator": "gt", + "actual_fraction": 0.8090804926955028, + "metric": 0.9050415353766829, + "rescued": 1123, + "harmed": 377, + "missed_rescue": 31, + "avoided_harm": 31, + "selected_both_correct": 3903, + "selected_both_wrong": 246 + }, + { + "budget": 0.82, + "threshold": -0.9996140599250793, + "operator": "gt", + "actual_fraction": 0.8198224004583214, + "metric": 0.905327986250358, + "rescued": 1127, + "harmed": 379, + "missed_rescue": 27, + "avoided_harm": 29, + "selected_both_correct": 3972, + "selected_both_wrong": 246 + }, + { + "budget": 0.83, + "threshold": -0.9996429681777954, + "operator": "gt", + "actual_fraction": 0.8281294757949011, + "metric": 0.9051847608135205, + "rescued": 1128, + "harmed": 381, + "missed_rescue": 26, + "avoided_harm": 27, + "selected_both_correct": 4027, + "selected_both_wrong": 246 + }, + { + "budget": 0.84, + "threshold": -0.9996750354766846, + "operator": "gt", + "actual_fraction": 0.8395875107419077, + "metric": 0.9054712116871957, + "rescued": 1131, + "harmed": 382, + "missed_rescue": 23, + "avoided_harm": 26, + "selected_both_correct": 4102, + "selected_both_wrong": 247 + }, + { + "budget": 0.85, + "threshold": -0.9997040629386902, + "operator": "gt", + "actual_fraction": 0.8458894299627614, + "metric": 0.9056144371240332, + "rescued": 1132, + "harmed": 382, + "missed_rescue": 22, + "avoided_harm": 26, + "selected_both_correct": 4143, + "selected_both_wrong": 249 + }, + { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8587797192781438, + "metric": 0.9057576625608708, + "rescued": 1134, + "harmed": 383, + "missed_rescue": 20, + "avoided_harm": 25, + "selected_both_correct": 4230, + "selected_both_wrong": 249 + }, + { + "budget": 0.87, + "threshold": -0.9997695088386536, + "operator": "gt", + "actual_fraction": 0.8693784016041249, + "metric": 0.9057576625608708, + "rescued": 1136, + "harmed": 385, + "missed_rescue": 18, + "avoided_harm": 23, + "selected_both_correct": 4300, + "selected_both_wrong": 249 + }, + { + "budget": 0.88, + "threshold": -0.999783456325531, + "operator": "gt", + "actual_fraction": 0.8749641936407906, + "metric": 0.9057576625608708, + "rescued": 1136, + "harmed": 385, + "missed_rescue": 18, + "avoided_harm": 23, + "selected_both_correct": 4339, + "selected_both_wrong": 249 + }, + { + "budget": 0.89, + "threshold": -0.9998205304145813, + "operator": "gt", + "actual_fraction": 0.8894299627613864, + "metric": 0.905327986250358, + "rescued": 1137, + "harmed": 389, + "missed_rescue": 17, + "avoided_harm": 19, + "selected_both_correct": 4434, + "selected_both_wrong": 250 + }, + { + "budget": 0.9, + "threshold": -0.999841570854187, + "operator": "gt", + "actual_fraction": 0.8988828415926668, + "metric": 0.905327986250358, + "rescued": 1140, + "harmed": 392, + "missed_rescue": 14, + "avoided_harm": 16, + "selected_both_correct": 4494, + "selected_both_wrong": 250 + }, + { + "budget": 0.91, + "threshold": -0.9998601675033569, + "operator": "gt", + "actual_fraction": 0.9093382984818104, + "metric": 0.9054712116871957, + "rescued": 1143, + "harmed": 394, + "missed_rescue": 11, + "avoided_harm": 14, + "selected_both_correct": 4562, + "selected_both_wrong": 250 + }, + { + "budget": 0.92, + "threshold": -0.9998766183853149, + "operator": "gt", + "actual_fraction": 0.9185047264394156, + "metric": 0.9051847608135205, + "rescued": 1144, + "harmed": 397, + "missed_rescue": 10, + "avoided_harm": 11, + "selected_both_correct": 4620, + "selected_both_wrong": 252 + }, + { + "budget": 0.93, + "threshold": -0.9998977184295654, + "operator": "gt", + "actual_fraction": 0.9276711543970209, + "metric": 0.905327986250358, + "rescued": 1145, + "harmed": 397, + "missed_rescue": 9, + "avoided_harm": 11, + "selected_both_correct": 4683, + "selected_both_wrong": 252 + }, + { + "budget": 0.94, + "threshold": -0.9999151229858398, + "operator": "gt", + "actual_fraction": 0.9394156402177026, + "metric": 0.905327986250358, + "rescued": 1147, + "harmed": 399, + "missed_rescue": 7, + "avoided_harm": 9, + "selected_both_correct": 4760, + "selected_both_wrong": 253 + }, + { + "budget": 0.95, + "threshold": -0.999931812286377, + "operator": "gt", + "actual_fraction": 0.9497278716700086, + "metric": 0.9051847608135205, + "rescued": 1148, + "harmed": 401, + "missed_rescue": 6, + "avoided_harm": 7, + "selected_both_correct": 4829, + "selected_both_wrong": 253 + }, + { + "budget": 0.96, + "threshold": -0.9999452829360962, + "operator": "gt", + "actual_fraction": 0.9590375250644515, + "metric": 0.9054712116871957, + "rescued": 1151, + "harmed": 402, + "missed_rescue": 3, + "avoided_harm": 6, + "selected_both_correct": 4889, + "selected_both_wrong": 254 + }, + { + "budget": 0.97, + "threshold": -0.9999599456787109, + "operator": "gt", + "actual_fraction": 0.969492981953595, + "metric": 0.9050415353766829, + "rescued": 1151, + "harmed": 405, + "missed_rescue": 3, + "avoided_harm": 3, + "selected_both_correct": 4959, + "selected_both_wrong": 254 + }, + { + "budget": 0.98, + "threshold": -0.9999687671661377, + "operator": "gt", + "actual_fraction": 0.9783729590375251, + "metric": 0.9047550845030078, + "rescued": 1151, + "harmed": 407, + "missed_rescue": 3, + "avoided_harm": 1, + "selected_both_correct": 5019, + "selected_both_wrong": 254 + }, + { + "budget": 0.99, + "threshold": -0.9999842643737793, + "operator": "gt", + "actual_fraction": 0.9898309939845317, + "metric": 0.9048983099398453, + "rescued": 1153, + "harmed": 408, + "missed_rescue": 1, + "avoided_harm": 0, + "selected_both_correct": 5096, + "selected_both_wrong": 254 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9050415353766829, + "rescued": 1154, + "harmed": 408, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 5165, + "selected_both_wrong": 255 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.8000287005811868, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2247, + "avoided_harm": 845, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5245773792266846, + "operator": "gt", + "actual_fraction": 0.011623735380641457, + "metric": 0.8039032790414006, + "rescued": 70, + "harmed": 16, + "missed_rescue": 2177, + "avoided_harm": 829, + "selected_both_correct": 66, + "selected_both_wrong": 10 + }, + { + "budget": 0.02, + "threshold": -0.5456489324569702, + "operator": "gt", + "actual_fraction": 0.021453684437109852, + "metric": 0.8067733371600775, + "rescued": 122, + "harmed": 28, + "missed_rescue": 2125, + "avoided_harm": 817, + "selected_both_correct": 124, + "selected_both_wrong": 25 + }, + { + "budget": 0.03, + "threshold": -0.5652987360954285, + "operator": "ge", + "actual_fraction": 0.02970510152830595, + "metric": 0.8093563894668867, + "rescued": 169, + "harmed": 39, + "missed_rescue": 2078, + "avoided_harm": 806, + "selected_both_correct": 169, + "selected_both_wrong": 37 + }, + { + "budget": 0.04, + "threshold": -0.5851011276245117, + "operator": "ge", + "actual_fraction": 0.03795651861950204, + "metric": 0.8113654301499605, + "rescued": 209, + "harmed": 51, + "missed_rescue": 2038, + "avoided_harm": 794, + "selected_both_correct": 224, + "selected_both_wrong": 45 + }, + { + "budget": 0.05, + "threshold": -0.6055658459663391, + "operator": "ge", + "actual_fraction": 0.047212456052235056, + "metric": 0.8140202339097367, + "rescued": 259, + "harmed": 64, + "missed_rescue": 1988, + "avoided_harm": 781, + "selected_both_correct": 283, + "selected_both_wrong": 52 + }, + { + "budget": 0.06, + "threshold": -0.6293190717697144, + "operator": "ge", + "actual_fraction": 0.057759919638372674, + "metric": 0.817464303652149, + "rescued": 318, + "harmed": 75, + "missed_rescue": 1929, + "avoided_harm": 770, + "selected_both_correct": 345, + "selected_both_wrong": 67 + }, + { + "budget": 0.07, + "threshold": -0.6477982401847839, + "operator": "ge", + "actual_fraction": 0.06601133672956877, + "metric": 0.8201908588648921, + "rescued": 365, + "harmed": 84, + "missed_rescue": 1882, + "avoided_harm": 761, + "selected_both_correct": 399, + "selected_both_wrong": 72 + }, + { + "budget": 0.08, + "threshold": -0.6693127155303955, + "operator": "ge", + "actual_fraction": 0.07483676544450026, + "metric": 0.8232761713424697, + "rescued": 417, + "harmed": 93, + "missed_rescue": 1830, + "avoided_harm": 752, + "selected_both_correct": 451, + "selected_both_wrong": 82 + }, + { + "budget": 0.09, + "threshold": -0.6888871788978577, + "operator": "gt", + "actual_fraction": 0.08488196885986941, + "metric": 0.8261462294611466, + "rescued": 464, + "harmed": 100, + "missed_rescue": 1783, + "avoided_harm": 745, + "selected_both_correct": 524, + "selected_both_wrong": 95 + }, + { + "budget": 0.1, + "threshold": -0.7061904072761536, + "operator": "gt", + "actual_fraction": 0.09356389466886704, + "metric": 0.8290162875798235, + "rescued": 511, + "harmed": 107, + "missed_rescue": 1736, + "avoided_harm": 738, + "selected_both_correct": 578, + "selected_both_wrong": 108 + }, + { + "budget": 0.11, + "threshold": -0.7233095169067383, + "operator": "ge", + "actual_fraction": 0.1026045777426993, + "metric": 0.8319580971514673, + "rescued": 558, + "harmed": 113, + "missed_rescue": 1689, + "avoided_harm": 732, + "selected_both_correct": 643, + "selected_both_wrong": 116 + }, + { + "budget": 0.12, + "threshold": -0.7424216866493225, + "operator": "ge", + "actual_fraction": 0.11322379278180383, + "metric": 0.8348999067231111, + "rescued": 617, + "harmed": 131, + "missed_rescue": 1630, + "avoided_harm": 714, + "selected_both_correct": 708, + "selected_both_wrong": 122 + }, + { + "budget": 0.13, + "threshold": -0.7627776861190796, + "operator": "gt", + "actual_fraction": 0.12255148166750376, + "metric": 0.8375547104828873, + "rescued": 664, + "harmed": 141, + "missed_rescue": 1583, + "avoided_harm": 704, + "selected_both_correct": 775, + "selected_both_wrong": 128 + }, + { + "budget": 0.14, + "threshold": -0.7786493301391602, + "operator": "gt", + "actual_fraction": 0.13137691038243524, + "metric": 0.839635502618928, + "rescued": 705, + "harmed": 153, + "missed_rescue": 1542, + "avoided_harm": 692, + "selected_both_correct": 828, + "selected_both_wrong": 145 + }, + { + "budget": 0.15, + "threshold": -0.7950222492218018, + "operator": "ge", + "actual_fraction": 0.14056109636220132, + "metric": 0.8421468034727704, + "rescued": 753, + "harmed": 166, + "missed_rescue": 1494, + "avoided_harm": 679, + "selected_both_correct": 884, + "selected_both_wrong": 156 + }, + { + "budget": 0.16, + "threshold": -0.8140527606010437, + "operator": "ge", + "actual_fraction": 0.15096505704240512, + "metric": 0.8448733586855134, + "rescued": 804, + "harmed": 179, + "missed_rescue": 1443, + "avoided_harm": 666, + "selected_both_correct": 955, + "selected_both_wrong": 166 + }, + { + "budget": 0.17, + "threshold": -0.8327813148498535, + "operator": "ge", + "actual_fraction": 0.1613690177226089, + "metric": 0.8468823993685872, + "rescued": 844, + "harmed": 191, + "missed_rescue": 1403, + "avoided_harm": 654, + "selected_both_correct": 1033, + "selected_both_wrong": 181 + }, + { + "budget": 0.18, + "threshold": -0.8464508652687073, + "operator": "gt", + "actual_fraction": 0.1702661978905073, + "metric": 0.8490349429575949, + "rescued": 882, + "harmed": 199, + "missed_rescue": 1365, + "avoided_harm": 646, + "selected_both_correct": 1102, + "selected_both_wrong": 190 + }, + { + "budget": 0.19, + "threshold": -0.858244001865387, + "operator": "ge", + "actual_fraction": 0.17916337805840568, + "metric": 0.851761498170338, + "rescued": 932, + "harmed": 211, + "missed_rescue": 1315, + "avoided_harm": 634, + "selected_both_correct": 1158, + "selected_both_wrong": 196 + }, + { + "budget": 0.2, + "threshold": -0.8701565861701965, + "operator": "ge", + "actual_fraction": 0.18798880677333715, + "metric": 0.8539857932123125, + "rescued": 972, + "harmed": 220, + "missed_rescue": 1275, + "avoided_harm": 625, + "selected_both_correct": 1228, + "selected_both_wrong": 200 + }, + { + "budget": 0.21, + "threshold": -0.8840392827987671, + "operator": "gt", + "actual_fraction": 0.19659898112936788, + "metric": 0.855420822271651, + "rescued": 1004, + "harmed": 232, + "missed_rescue": 1243, + "avoided_harm": 613, + "selected_both_correct": 1293, + "selected_both_wrong": 211 + }, + { + "budget": 0.22, + "threshold": -0.8933094143867493, + "operator": "ge", + "actual_fraction": 0.20642893018583627, + "metric": 0.8580038745784602, + "rescued": 1047, + "harmed": 239, + "missed_rescue": 1200, + "avoided_harm": 606, + "selected_both_correct": 1373, + "selected_both_wrong": 218 + }, + { + "budget": 0.23, + "threshold": -0.9046505093574524, + "operator": "gt", + "actual_fraction": 0.21711989667790774, + "metric": 0.860012915261534, + "rescued": 1088, + "harmed": 252, + "missed_rescue": 1159, + "avoided_harm": 593, + "selected_both_correct": 1453, + "selected_both_wrong": 233 + }, + { + "budget": 0.24, + "threshold": -0.9136765599250793, + "operator": "ge", + "actual_fraction": 0.22766736026404535, + "metric": 0.8627394704742771, + "rescued": 1134, + "harmed": 260, + "missed_rescue": 1113, + "avoided_harm": 585, + "selected_both_correct": 1534, + "selected_both_wrong": 245 + }, + { + "budget": 0.25, + "threshold": -0.9230391979217529, + "operator": "gt", + "actual_fraction": 0.23785606658534836, + "metric": 0.865609528592954, + "rescued": 1183, + "harmed": 269, + "missed_rescue": 1064, + "avoided_harm": 576, + "selected_both_correct": 1610, + "selected_both_wrong": 253 + }, + { + "budget": 0.26, + "threshold": -0.9314625263214111, + "operator": "gt", + "actual_fraction": 0.24804477290665136, + "metric": 0.8672598120111932, + "rescued": 1221, + "harmed": 284, + "missed_rescue": 1026, + "avoided_harm": 561, + "selected_both_correct": 1685, + "selected_both_wrong": 267 + }, + { + "budget": 0.27, + "threshold": -0.9376689791679382, + "operator": "ge", + "actual_fraction": 0.25751596469828514, + "metric": 0.8686230896175647, + "rescued": 1253, + "harmed": 297, + "missed_rescue": 994, + "avoided_harm": 548, + "selected_both_correct": 1760, + "selected_both_wrong": 279 + }, + { + "budget": 0.28, + "threshold": -0.9449946880340576, + "operator": "gt", + "actual_fraction": 0.26863743990815814, + "metric": 0.8702016215828371, + "rescued": 1285, + "harmed": 307, + "missed_rescue": 962, + "avoided_harm": 538, + "selected_both_correct": 1858, + "selected_both_wrong": 294 + }, + { + "budget": 0.29, + "threshold": -0.9511421918869019, + "operator": "gt", + "actual_fraction": 0.2793284064002296, + "metric": 0.872067159359977, + "rescued": 1328, + "harmed": 324, + "missed_rescue": 919, + "avoided_harm": 521, + "selected_both_correct": 1938, + "selected_both_wrong": 303 + }, + { + "budget": 0.3, + "threshold": -0.9559813141822815, + "operator": "gt", + "actual_fraction": 0.2897323670804334, + "metric": 0.8733586855133817, + "rescued": 1356, + "harmed": 334, + "missed_rescue": 891, + "avoided_harm": 511, + "selected_both_correct": 2036, + "selected_both_wrong": 312 + }, + { + "budget": 0.31, + "threshold": -0.9597620368003845, + "operator": "ge", + "actual_fraction": 0.2999928248547033, + "metric": 0.8741479514960178, + "rescued": 1382, + "harmed": 349, + "missed_rescue": 865, + "avoided_harm": 496, + "selected_both_correct": 2133, + "selected_both_wrong": 317 + }, + { + "budget": 0.32, + "threshold": -0.9643214344978333, + "operator": "gt", + "actual_fraction": 0.3091770108344694, + "metric": 0.8758699863672239, + "rescued": 1416, + "harmed": 359, + "missed_rescue": 831, + "avoided_harm": 486, + "selected_both_correct": 2209, + "selected_both_wrong": 325 + }, + { + "budget": 0.33, + "threshold": -0.9686195850372314, + "operator": "gt", + "actual_fraction": 0.3218770180096147, + "metric": 0.8774485183324963, + "rescued": 1451, + "harmed": 372, + "missed_rescue": 796, + "avoided_harm": 473, + "selected_both_correct": 2333, + "selected_both_wrong": 330 + }, + { + "budget": 0.34, + "threshold": -0.9719924926757812, + "operator": "gt", + "actual_fraction": 0.33127645834828157, + "metric": 0.8790988017507354, + "rescued": 1484, + "harmed": 382, + "missed_rescue": 763, + "avoided_harm": 463, + "selected_both_correct": 2416, + "selected_both_wrong": 335 + }, + { + "budget": 0.35, + "threshold": -0.974821150302887, + "operator": "ge", + "actual_fraction": 0.3431871995407907, + "metric": 0.8801750735452393, + "rescued": 1514, + "harmed": 397, + "missed_rescue": 733, + "avoided_harm": 448, + "selected_both_correct": 2529, + "selected_both_wrong": 343 + }, + { + "budget": 0.36, + "threshold": -0.9777138829231262, + "operator": "gt", + "actual_fraction": 0.35280189423835834, + "metric": 0.8812513453397431, + "rescued": 1539, + "harmed": 407, + "missed_rescue": 708, + "avoided_harm": 438, + "selected_both_correct": 2623, + "selected_both_wrong": 348 + }, + { + "budget": 0.37, + "threshold": -0.9802808165550232, + "operator": "gt", + "actual_fraction": 0.36421037526009903, + "metric": 0.8827581258520485, + "rescued": 1569, + "harmed": 416, + "missed_rescue": 678, + "avoided_harm": 429, + "selected_both_correct": 2740, + "selected_both_wrong": 351 + }, + { + "budget": 0.38, + "threshold": -0.982557475566864, + "operator": "gt", + "actual_fraction": 0.3761928679055751, + "metric": 0.8843366578173208, + "rescued": 1602, + "harmed": 427, + "missed_rescue": 645, + "avoided_harm": 418, + "selected_both_correct": 2856, + "selected_both_wrong": 358 + }, + { + "budget": 0.39, + "threshold": -0.9843363761901855, + "operator": "gt", + "actual_fraction": 0.38803185764511733, + "metric": 0.8857716868766593, + "rescued": 1630, + "harmed": 435, + "missed_rescue": 617, + "avoided_harm": 410, + "selected_both_correct": 2978, + "selected_both_wrong": 365 + }, + { + "budget": 0.4, + "threshold": -0.9863631129264832, + "operator": "gt", + "actual_fraction": 0.4008036162732295, + "metric": 0.8869197101241301, + "rescued": 1658, + "harmed": 447, + "missed_rescue": 589, + "avoided_harm": 398, + "selected_both_correct": 3105, + "selected_both_wrong": 376 + }, + { + "budget": 0.41, + "threshold": -0.9879462122917175, + "operator": "gt", + "actual_fraction": 0.41156633421826794, + "metric": 0.8880677333716007, + "rescued": 1685, + "harmed": 458, + "missed_rescue": 562, + "avoided_harm": 387, + "selected_both_correct": 3212, + "selected_both_wrong": 381 + }, + { + "budget": 0.42, + "threshold": -0.9890130758285522, + "operator": "gt", + "actual_fraction": 0.4204635143861663, + "metric": 0.8884264906364354, + "rescued": 1701, + "harmed": 469, + "missed_rescue": 546, + "avoided_harm": 376, + "selected_both_correct": 3303, + "selected_both_wrong": 387 + }, + { + "budget": 0.43, + "threshold": -0.9899863600730896, + "operator": "gt", + "actual_fraction": 0.4293606945540647, + "metric": 0.8889287508072038, + "rescued": 1720, + "harmed": 481, + "missed_rescue": 527, + "avoided_harm": 364, + "selected_both_correct": 3391, + "selected_both_wrong": 392 + }, + { + "budget": 0.44, + "threshold": -0.9911527037620544, + "operator": "gt", + "actual_fraction": 0.44119968429360695, + "metric": 0.890579034225443, + "rescued": 1755, + "harmed": 493, + "missed_rescue": 492, + "avoided_harm": 352, + "selected_both_correct": 3502, + "selected_both_wrong": 399 + }, + { + "budget": 0.45, + "threshold": -0.9919379353523254, + "operator": "gt", + "actual_fraction": 0.4516753964267776, + "metric": 0.8913683002080792, + "rescued": 1778, + "harmed": 505, + "missed_rescue": 469, + "avoided_harm": 340, + "selected_both_correct": 3608, + "selected_both_wrong": 404 + }, + { + "budget": 0.46, + "threshold": -0.9926542043685913, + "operator": "gt", + "actual_fraction": 0.4611465882184114, + "metric": 0.89251632345555, + "rescued": 1797, + "harmed": 508, + "missed_rescue": 450, + "avoided_harm": 337, + "selected_both_correct": 3715, + "selected_both_wrong": 407 + }, + { + "budget": 0.47, + "threshold": -0.9933071732521057, + "operator": "gt", + "actual_fraction": 0.4706895314630121, + "metric": 0.8933055894381862, + "rescued": 1814, + "harmed": 514, + "missed_rescue": 433, + "avoided_harm": 331, + "selected_both_correct": 3823, + "selected_both_wrong": 409 + }, + { + "budget": 0.48, + "threshold": -0.9939024448394775, + "operator": "gt", + "actual_fraction": 0.481021740690249, + "metric": 0.8941666068737892, + "rescued": 1835, + "harmed": 523, + "missed_rescue": 412, + "avoided_harm": 322, + "selected_both_correct": 3932, + "selected_both_wrong": 414 + }, + { + "budget": 0.49, + "threshold": -0.9944451451301575, + "operator": "gt", + "actual_fraction": 0.4912821984645189, + "metric": 0.8946688670445576, + "rescued": 1853, + "harmed": 534, + "missed_rescue": 394, + "avoided_harm": 311, + "selected_both_correct": 4037, + "selected_both_wrong": 423 + }, + { + "budget": 0.5, + "threshold": -0.9949396848678589, + "operator": "gt", + "actual_fraction": 0.4992466097438473, + "metric": 0.89531463012126, + "rescued": 1869, + "harmed": 541, + "missed_rescue": 378, + "avoided_harm": 304, + "selected_both_correct": 4122, + "selected_both_wrong": 426 + }, + { + "budget": 0.51, + "threshold": -0.9952448010444641, + "operator": "gt", + "actual_fraction": 0.5059912463227381, + "metric": 0.8957451388390615, + "rescued": 1882, + "harmed": 548, + "missed_rescue": 365, + "avoided_harm": 297, + "selected_both_correct": 4196, + "selected_both_wrong": 426 + }, + { + "budget": 0.52, + "threshold": -0.9956685304641724, + "operator": "gt", + "actual_fraction": 0.5140991605080003, + "metric": 0.8960321446509292, + "rescued": 1894, + "harmed": 556, + "missed_rescue": 353, + "avoided_harm": 289, + "selected_both_correct": 4286, + "selected_both_wrong": 429 + }, + { + "budget": 0.53, + "threshold": -0.9961755275726318, + "operator": "gt", + "actual_fraction": 0.529166965631054, + "metric": 0.8973954222573007, + "rescued": 1921, + "harmed": 564, + "missed_rescue": 326, + "avoided_harm": 281, + "selected_both_correct": 4451, + "selected_both_wrong": 439 + }, + { + "budget": 0.54, + "threshold": -0.9965165853500366, + "operator": "gt", + "actual_fraction": 0.5406471981057617, + "metric": 0.8981129367869699, + "rescued": 1940, + "harmed": 573, + "missed_rescue": 307, + "avoided_harm": 272, + "selected_both_correct": 4575, + "selected_both_wrong": 447 + }, + { + "budget": 0.55, + "threshold": -0.9968273043632507, + "operator": "gt", + "actual_fraction": 0.5504771471622301, + "metric": 0.8979694338810361, + "rescued": 1947, + "harmed": 582, + "missed_rescue": 300, + "avoided_harm": 263, + "selected_both_correct": 4695, + "selected_both_wrong": 448 + }, + { + "budget": 0.56, + "threshold": -0.9971103668212891, + "operator": "gt", + "actual_fraction": 0.5621726339958384, + "metric": 0.8986151969577384, + "rescued": 1966, + "harmed": 592, + "missed_rescue": 281, + "avoided_harm": 253, + "selected_both_correct": 4825, + "selected_both_wrong": 452 + }, + { + "budget": 0.57, + "threshold": -0.997285008430481, + "operator": "gt", + "actual_fraction": 0.5677692473272584, + "metric": 0.898973954222573, + "rescued": 1977, + "harmed": 598, + "missed_rescue": 270, + "avoided_harm": 247, + "selected_both_correct": 4884, + "selected_both_wrong": 454 + }, + { + "budget": 0.58, + "threshold": -0.9974491000175476, + "operator": "gt", + "actual_fraction": 0.5759489129654876, + "metric": 0.8992609600344407, + "rescued": 1990, + "harmed": 607, + "missed_rescue": 257, + "avoided_harm": 238, + "selected_both_correct": 4973, + "selected_both_wrong": 457 + }, + { + "budget": 0.59, + "threshold": -0.9976769089698792, + "operator": "gt", + "actual_fraction": 0.5864963765516251, + "metric": 0.899834971658176, + "rescued": 2008, + "harmed": 617, + "missed_rescue": 239, + "avoided_harm": 228, + "selected_both_correct": 5091, + "selected_both_wrong": 458 + }, + { + "budget": 0.6, + "threshold": -0.9978172779083252, + "operator": "gt", + "actual_fraction": 0.5942455334720528, + "metric": 0.899834971658176, + "rescued": 2016, + "harmed": 625, + "missed_rescue": 231, + "avoided_harm": 220, + "selected_both_correct": 5179, + "selected_both_wrong": 462 + }, + { + "budget": 0.61, + "threshold": -0.9980121850967407, + "operator": "gt", + "actual_fraction": 0.6054387601348927, + "metric": 0.8994044629403746, + "rescued": 2025, + "harmed": 640, + "missed_rescue": 222, + "avoided_harm": 205, + "selected_both_correct": 5308, + "selected_both_wrong": 465 + }, + { + "budget": 0.62, + "threshold": -0.9981897473335266, + "operator": "gt", + "actual_fraction": 0.6165602353447658, + "metric": 0.9001219774700437, + "rescued": 2039, + "harmed": 644, + "missed_rescue": 208, + "avoided_harm": 201, + "selected_both_correct": 5438, + "selected_both_wrong": 472 + }, + { + "budget": 0.63, + "threshold": -0.9982993006706238, + "operator": "gt", + "actual_fraction": 0.6242376408122264, + "metric": 0.9004089832819114, + "rescued": 2049, + "harmed": 650, + "missed_rescue": 198, + "avoided_harm": 195, + "selected_both_correct": 5528, + "selected_both_wrong": 473 + }, + { + "budget": 0.64, + "threshold": -0.9984021782875061, + "operator": "gt", + "actual_fraction": 0.6322020520915549, + "metric": 0.900767740546746, + "rescued": 2058, + "harmed": 654, + "missed_rescue": 189, + "avoided_harm": 191, + "selected_both_correct": 5624, + "selected_both_wrong": 475 + }, + { + "budget": 0.65, + "threshold": -0.9985449314117432, + "operator": "gt", + "actual_fraction": 0.6454043194374686, + "metric": 0.9011982492645476, + "rescued": 2072, + "harmed": 662, + "missed_rescue": 175, + "avoided_harm": 183, + "selected_both_correct": 5784, + "selected_both_wrong": 477 + }, + { + "budget": 0.66, + "threshold": -0.9986749291419983, + "operator": "gt", + "actual_fraction": 0.6566692975532754, + "metric": 0.9017005094353161, + "rescued": 2085, + "harmed": 668, + "missed_rescue": 162, + "avoided_harm": 177, + "selected_both_correct": 5920, + "selected_both_wrong": 479 + }, + { + "budget": 0.67, + "threshold": -0.9987550973892212, + "operator": "gt", + "actual_fraction": 0.6646337088326039, + "metric": 0.9017005094353161, + "rescued": 2089, + "harmed": 672, + "missed_rescue": 158, + "avoided_harm": 173, + "selected_both_correct": 6021, + "selected_both_wrong": 481 + }, + { + "budget": 0.68, + "threshold": -0.9988664388656616, + "operator": "gt", + "actual_fraction": 0.6776207218196169, + "metric": 0.9021310181531176, + "rescued": 2102, + "harmed": 679, + "missed_rescue": 145, + "avoided_harm": 166, + "selected_both_correct": 6181, + "selected_both_wrong": 482 + }, + { + "budget": 0.69, + "threshold": -0.9989677667617798, + "operator": "gt", + "actual_fraction": 0.690464231900696, + "metric": 0.9020592667001507, + "rescued": 2113, + "harmed": 691, + "missed_rescue": 134, + "avoided_harm": 154, + "selected_both_correct": 6336, + "selected_both_wrong": 483 + }, + { + "budget": 0.7, + "threshold": -0.9990302324295044, + "operator": "gt", + "actual_fraction": 0.7000789265982637, + "metric": 0.9024180239649853, + "rescued": 2123, + "harmed": 696, + "missed_rescue": 124, + "avoided_harm": 149, + "selected_both_correct": 6453, + "selected_both_wrong": 485 + }, + { + "budget": 0.71, + "threshold": -0.9991169571876526, + "operator": "gt", + "actual_fraction": 0.7121331706967066, + "metric": 0.901772260888283, + "rescued": 2130, + "harmed": 712, + "missed_rescue": 117, + "avoided_harm": 133, + "selected_both_correct": 6595, + "selected_both_wrong": 488 + }, + { + "budget": 0.72, + "threshold": -0.9991704225540161, + "operator": "gt", + "actual_fraction": 0.7201693334290019, + "metric": 0.9017005094353161, + "rescued": 2136, + "harmed": 719, + "missed_rescue": 111, + "avoided_harm": 126, + "selected_both_correct": 6693, + "selected_both_wrong": 489 + }, + { + "budget": 0.73, + "threshold": -0.9992445707321167, + "operator": "gt", + "actual_fraction": 0.7310755542799742, + "metric": 0.9020592667001507, + "rescued": 2143, + "harmed": 721, + "missed_rescue": 104, + "avoided_harm": 124, + "selected_both_correct": 6833, + "selected_both_wrong": 492 + }, + { + "budget": 0.74, + "threshold": -0.9992903470993042, + "operator": "gt", + "actual_fraction": 0.7394704742771041, + "metric": 0.9018440123412499, + "rescued": 2147, + "harmed": 728, + "missed_rescue": 100, + "avoided_harm": 117, + "selected_both_correct": 6937, + "selected_both_wrong": 494 + }, + { + "budget": 0.75, + "threshold": -0.9993333220481873, + "operator": "gt", + "actual_fraction": 0.7467173710267633, + "metric": 0.9018440123412499, + "rescued": 2151, + "harmed": 732, + "missed_rescue": 96, + "avoided_harm": 113, + "selected_both_correct": 7028, + "selected_both_wrong": 496 + }, + { + "budget": 0.76, + "threshold": -0.999373733997345, + "operator": "gt", + "actual_fraction": 0.7550405395709263, + "metric": 0.9016287579823491, + "rescued": 2154, + "harmed": 738, + "missed_rescue": 93, + "avoided_harm": 107, + "selected_both_correct": 7133, + "selected_both_wrong": 498 + }, + { + "budget": 0.77, + "threshold": -0.9994296431541443, + "operator": "gt", + "actual_fraction": 0.7659467604218986, + "metric": 0.9018440123412499, + "rescued": 2164, + "harmed": 745, + "missed_rescue": 83, + "avoided_harm": 100, + "selected_both_correct": 7267, + "selected_both_wrong": 499 + }, + { + "budget": 0.78, + "threshold": -0.999464213848114, + "operator": "gt", + "actual_fraction": 0.7743416804190285, + "metric": 0.9022027696060845, + "rescued": 2172, + "harmed": 748, + "missed_rescue": 75, + "avoided_harm": 97, + "selected_both_correct": 7371, + "selected_both_wrong": 501 + }, + { + "budget": 0.79, + "threshold": -0.9994966983795166, + "operator": "gt", + "actual_fraction": 0.7820908373394562, + "metric": 0.9022745210590515, + "rescued": 2178, + "harmed": 753, + "missed_rescue": 69, + "avoided_harm": 92, + "selected_both_correct": 7467, + "selected_both_wrong": 502 + }, + { + "budget": 0.8, + "threshold": -0.9995417594909668, + "operator": "gt", + "actual_fraction": 0.7935710698141637, + "metric": 0.9023462725120184, + "rescued": 2182, + "harmed": 756, + "missed_rescue": 65, + "avoided_harm": 89, + "selected_both_correct": 7616, + "selected_both_wrong": 506 + }, + { + "budget": 0.81, + "threshold": -0.9995827078819275, + "operator": "gt", + "actual_fraction": 0.8052665566477721, + "metric": 0.9025615268709192, + "rescued": 2189, + "harmed": 760, + "missed_rescue": 58, + "avoided_harm": 85, + "selected_both_correct": 7765, + "selected_both_wrong": 509 + }, + { + "budget": 0.82, + "threshold": -0.9996140599250793, + "operator": "gt", + "actual_fraction": 0.8185405754466528, + "metric": 0.9026332783238861, + "rescued": 2196, + "harmed": 766, + "missed_rescue": 51, + "avoided_harm": 79, + "selected_both_correct": 7933, + "selected_both_wrong": 513 + }, + { + "budget": 0.83, + "threshold": -0.9996429681777954, + "operator": "gt", + "actual_fraction": 0.8264332352730143, + "metric": 0.9026332783238861, + "rescued": 2200, + "harmed": 770, + "missed_rescue": 47, + "avoided_harm": 75, + "selected_both_correct": 8034, + "selected_both_wrong": 514 + }, + { + "budget": 0.84, + "threshold": -0.9996750354766846, + "operator": "gt", + "actual_fraction": 0.836837195953218, + "metric": 0.9025615268709192, + "rescued": 2202, + "harmed": 773, + "missed_rescue": 45, + "avoided_harm": 72, + "selected_both_correct": 8172, + "selected_both_wrong": 516 + }, + { + "budget": 0.85, + "threshold": -0.9997040629386902, + "operator": "gt", + "actual_fraction": 0.844299347061778, + "metric": 0.9025615268709192, + "rescued": 2206, + "harmed": 777, + "missed_rescue": 41, + "avoided_harm": 68, + "selected_both_correct": 8267, + "selected_both_wrong": 517 + }, + { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8580038745784602, + "metric": 0.9024180239649853, + "rescued": 2213, + "harmed": 786, + "missed_rescue": 34, + "avoided_harm": 59, + "selected_both_correct": 8437, + "selected_both_wrong": 522 + }, + { + "budget": 0.87, + "threshold": -0.9997695088386536, + "operator": "gt", + "actual_fraction": 0.870273373035804, + "metric": 0.9026332783238861, + "rescued": 2220, + "harmed": 790, + "missed_rescue": 27, + "avoided_harm": 55, + "selected_both_correct": 8594, + "selected_both_wrong": 525 + }, + { + "budget": 0.88, + "threshold": -0.999783456325531, + "operator": "gt", + "actual_fraction": 0.8750807203845878, + "metric": 0.9024180239649853, + "rescued": 2220, + "harmed": 793, + "missed_rescue": 27, + "avoided_harm": 52, + "selected_both_correct": 8657, + "selected_both_wrong": 526 + }, + { + "budget": 0.89, + "threshold": -0.9998205304145813, + "operator": "gt", + "actual_fraction": 0.8908660400373107, + "metric": 0.9022745210590515, + "rescued": 2223, + "harmed": 798, + "missed_rescue": 24, + "avoided_harm": 47, + "selected_both_correct": 8865, + "selected_both_wrong": 530 + }, + { + "budget": 0.9, + "threshold": -0.999841570854187, + "operator": "gt", + "actual_fraction": 0.8992609600344407, + "metric": 0.9023462725120184, + "rescued": 2225, + "harmed": 799, + "missed_rescue": 22, + "avoided_harm": 46, + "selected_both_correct": 8979, + "selected_both_wrong": 530 + }, + { + "budget": 0.91, + "threshold": -0.9998601675033569, + "operator": "gt", + "actual_fraction": 0.9091626605438761, + "metric": 0.9023462725120184, + "rescued": 2227, + "harmed": 801, + "missed_rescue": 20, + "avoided_harm": 44, + "selected_both_correct": 9111, + "selected_both_wrong": 532 + }, + { + "budget": 0.92, + "threshold": -0.9998766183853149, + "operator": "gt", + "actual_fraction": 0.9171270718232044, + "metric": 0.9023462725120184, + "rescued": 2230, + "harmed": 804, + "missed_rescue": 17, + "avoided_harm": 41, + "selected_both_correct": 9215, + "selected_both_wrong": 533 + }, + { + "budget": 0.93, + "threshold": -0.9998977184295654, + "operator": "gt", + "actual_fraction": 0.9273875295974744, + "metric": 0.9020592667001507, + "rescued": 2232, + "harmed": 810, + "missed_rescue": 15, + "avoided_harm": 35, + "selected_both_correct": 9350, + "selected_both_wrong": 533 + }, + { + "budget": 0.94, + "threshold": -0.9999151229858398, + "operator": "gt", + "actual_fraction": 0.9375044844658105, + "metric": 0.9018440123412499, + "rescued": 2235, + "harmed": 816, + "missed_rescue": 12, + "avoided_harm": 29, + "selected_both_correct": 9481, + "selected_both_wrong": 534 + }, + { + "budget": 0.95, + "threshold": -0.999931812286377, + "operator": "gt", + "actual_fraction": 0.948984716940518, + "metric": 0.9016287579823491, + "rescued": 2239, + "harmed": 823, + "missed_rescue": 8, + "avoided_harm": 22, + "selected_both_correct": 9629, + "selected_both_wrong": 535 + }, + { + "budget": 0.96, + "threshold": -0.9999452829360962, + "operator": "gt", + "actual_fraction": 0.9576666427495156, + "metric": 0.9015570065293822, + "rescued": 2241, + "harmed": 826, + "missed_rescue": 6, + "avoided_harm": 19, + "selected_both_correct": 9745, + "selected_both_wrong": 535 + }, + { + "budget": 0.97, + "threshold": -0.9999599456787109, + "operator": "gt", + "actual_fraction": 0.9709406615483963, + "metric": 0.9009829949056468, + "rescued": 2241, + "harmed": 834, + "missed_rescue": 6, + "avoided_harm": 11, + "selected_both_correct": 9922, + "selected_both_wrong": 535 + }, + { + "budget": 0.98, + "threshold": -0.9999687671661377, + "operator": "gt", + "actual_fraction": 0.9784745641099232, + "metric": 0.900839491999713, + "rescued": 2241, + "harmed": 836, + "missed_rescue": 6, + "avoided_harm": 9, + "selected_both_correct": 10023, + "selected_both_wrong": 537 + }, + { + "budget": 0.99, + "threshold": -0.9999842643737793, + "operator": "gt", + "actual_fraction": 0.98973954222573, + "metric": 0.900839491999713, + "rescued": 2245, + "harmed": 840, + "missed_rescue": 2, + "avoided_harm": 5, + "selected_both_correct": 10170, + "selected_both_wrong": 539 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9006242376408122, + "rescued": 2247, + "harmed": 845, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 10305, + "selected_both_wrong": 540 + } + ] + }, + "learned": { + "selected_budget_index": 77, + "validation_area_0_25": 0.8402320252076769, + "validation_point": { + "budget": 0.77, + "threshold": -3.183574676513672, + "operator": "ge", + "actual_fraction": 0.7699799484388428, + "metric": 0.909481523918648, + "rescued": 1066, + "harmed": 289, + "missed_rescue": 88, + "avoided_harm": 119, + "selected_both_correct": 3781, + "selected_both_wrong": 240 + }, + "test_point": { + "budget": 0.77, + "threshold": -3.183574676513672, + "operator": "ge", + "actual_fraction": 0.7665925234986009, + "metric": 0.9088039032790414, + "rescued": 2060, + "harmed": 544, + "missed_rescue": 187, + "avoided_harm": 301, + "selected_both_correct": 7581, + "selected_both_wrong": 499 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7981953594958464, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1154, + "avoided_harm": 408, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.821803569793701, + "operator": "ge", + "actual_fraction": 0.009882555141793183, + "metric": 0.8026353480378116, + "rescued": 33, + "harmed": 2, + "missed_rescue": 1121, + "avoided_harm": 406, + "selected_both_correct": 29, + "selected_both_wrong": 5 + }, + { + "budget": 0.02, + "threshold": 2.459282875061035, + "operator": "ge", + "actual_fraction": 0.01990833572042395, + "metric": 0.8077914637639645, + "rescued": 70, + "harmed": 3, + "missed_rescue": 1084, + "avoided_harm": 405, + "selected_both_correct": 58, + "selected_both_wrong": 8 + }, + { + "budget": 0.03, + "threshold": 2.2472119331359863, + "operator": "ge", + "actual_fraction": 0.029934116299054712, + "metric": 0.8118017759954168, + "rescued": 98, + "harmed": 3, + "missed_rescue": 1056, + "avoided_harm": 405, + "selected_both_correct": 96, + "selected_both_wrong": 12 + }, + { + "budget": 0.04, + "threshold": 2.0771026611328125, + "operator": "ge", + "actual_fraction": 0.039959896877685475, + "metric": 0.8166714408478946, + "rescued": 132, + "harmed": 3, + "missed_rescue": 1022, + "avoided_harm": 405, + "selected_both_correct": 127, + "selected_both_wrong": 17 + }, + { + "budget": 0.05, + "threshold": 1.8918991088867188, + "operator": "ge", + "actual_fraction": 0.04998567745631624, + "metric": 0.8205385276425093, + "rescued": 159, + "harmed": 3, + "missed_rescue": 995, + "avoided_harm": 405, + "selected_both_correct": 162, + "selected_both_wrong": 25 + }, + { + "budget": 0.06, + "threshold": 1.754787564277649, + "operator": "ge", + "actual_fraction": 0.059868232598109423, + "metric": 0.8238327126897736, + "rescued": 185, + "harmed": 6, + "missed_rescue": 969, + "avoided_harm": 402, + "selected_both_correct": 189, + "selected_both_wrong": 38 + }, + { + "budget": 0.07, + "threshold": 1.612305998802185, + "operator": "ge", + "actual_fraction": 0.06989401317674018, + "metric": 0.8272701231738757, + "rescued": 209, + "harmed": 6, + "missed_rescue": 945, + "avoided_harm": 402, + "selected_both_correct": 229, + "selected_both_wrong": 44 + }, + { + "budget": 0.08, + "threshold": 1.4815378189086914, + "operator": "ge", + "actual_fraction": 0.07991979375537095, + "metric": 0.830277857347465, + "rescued": 231, + "harmed": 7, + "missed_rescue": 923, + "avoided_harm": 401, + "selected_both_correct": 266, + "selected_both_wrong": 54 + }, + { + "budget": 0.09, + "threshold": 1.3550769090652466, + "operator": "ge", + "actual_fraction": 0.08994557433400172, + "metric": 0.8334288169578917, + "rescued": 254, + "harmed": 8, + "missed_rescue": 900, + "avoided_harm": 400, + "selected_both_correct": 299, + "selected_both_wrong": 67 + }, + { + "budget": 0.1, + "threshold": 1.2258758544921875, + "operator": "ge", + "actual_fraction": 0.09997135491263248, + "metric": 0.8360068748209682, + "rescued": 274, + "harmed": 10, + "missed_rescue": 880, + "avoided_harm": 398, + "selected_both_correct": 338, + "selected_both_wrong": 76 + }, + { + "budget": 0.11, + "threshold": 1.0788583755493164, + "operator": "ge", + "actual_fraction": 0.10999713549126325, + "metric": 0.839157834431395, + "rescued": 297, + "harmed": 11, + "missed_rescue": 857, + "avoided_harm": 397, + "selected_both_correct": 379, + "selected_both_wrong": 81 + }, + { + "budget": 0.12, + "threshold": 0.9625301957130432, + "operator": "ge", + "actual_fraction": 0.11987969063305644, + "metric": 0.8418791177313091, + "rescued": 319, + "harmed": 14, + "missed_rescue": 835, + "avoided_harm": 394, + "selected_both_correct": 419, + "selected_both_wrong": 85 + }, + { + "budget": 0.13, + "threshold": 0.8249785304069519, + "operator": "ge", + "actual_fraction": 0.1299054712116872, + "metric": 0.845316528215411, + "rescued": 346, + "harmed": 17, + "missed_rescue": 808, + "avoided_harm": 391, + "selected_both_correct": 452, + "selected_both_wrong": 92 + }, + { + "budget": 0.14, + "threshold": 0.7093843221664429, + "operator": "ge", + "actual_fraction": 0.13993125179031796, + "metric": 0.8477513606416499, + "rescued": 366, + "harmed": 20, + "missed_rescue": 788, + "avoided_harm": 388, + "selected_both_correct": 494, + "selected_both_wrong": 97 + }, + { + "budget": 0.15, + "threshold": 0.5911833047866821, + "operator": "ge", + "actual_fraction": 0.14995703236894872, + "metric": 0.8498997421942137, + "rescued": 384, + "harmed": 23, + "missed_rescue": 770, + "avoided_harm": 385, + "selected_both_correct": 538, + "selected_both_wrong": 102 + }, + { + "budget": 0.16, + "threshold": 0.5016208291053772, + "operator": "ge", + "actual_fraction": 0.1599828129475795, + "metric": 0.8526210254941278, + "rescued": 405, + "harmed": 25, + "missed_rescue": 749, + "avoided_harm": 383, + "selected_both_correct": 579, + "selected_both_wrong": 108 + }, + { + "budget": 0.17, + "threshold": 0.44066739082336426, + "operator": "ge", + "actual_fraction": 0.16986536808937266, + "metric": 0.8559152105413922, + "rescued": 432, + "harmed": 29, + "missed_rescue": 722, + "avoided_harm": 379, + "selected_both_correct": 613, + "selected_both_wrong": 112 + }, + { + "budget": 0.18, + "threshold": 0.37089839577674866, + "operator": "ge", + "actual_fraction": 0.17989114866800343, + "metric": 0.8577771412202807, + "rescued": 450, + "harmed": 34, + "missed_rescue": 704, + "avoided_harm": 374, + "selected_both_correct": 656, + "selected_both_wrong": 116 + }, + { + "budget": 0.19, + "threshold": 0.2769058346748352, + "operator": "ge", + "actual_fraction": 0.1899169292466342, + "metric": 0.8596390718991693, + "rescued": 468, + "harmed": 39, + "missed_rescue": 686, + "avoided_harm": 369, + "selected_both_correct": 698, + "selected_both_wrong": 121 + }, + { + "budget": 0.2, + "threshold": 0.20165908336639404, + "operator": "ge", + "actual_fraction": 0.19994270982526496, + "metric": 0.8606416499570324, + "rescued": 482, + "harmed": 46, + "missed_rescue": 672, + "avoided_harm": 362, + "selected_both_correct": 744, + "selected_both_wrong": 124 + }, + { + "budget": 0.21, + "threshold": 0.13794422149658203, + "operator": "ge", + "actual_fraction": 0.20996849040389573, + "metric": 0.8622171297622457, + "rescued": 502, + "harmed": 55, + "missed_rescue": 652, + "avoided_harm": 353, + "selected_both_correct": 783, + "selected_both_wrong": 126 + }, + { + "budget": 0.22, + "threshold": 0.06396147608757019, + "operator": "ge", + "actual_fraction": 0.2199942709825265, + "metric": 0.8629332569464336, + "rescued": 513, + "harmed": 61, + "missed_rescue": 641, + "avoided_harm": 347, + "selected_both_correct": 832, + "selected_both_wrong": 130 + }, + { + "budget": 0.23, + "threshold": -0.017305493354797363, + "operator": "ge", + "actual_fraction": 0.22987682612431967, + "metric": 0.8647951876253223, + "rescued": 530, + "harmed": 65, + "missed_rescue": 624, + "avoided_harm": 343, + "selected_both_correct": 876, + "selected_both_wrong": 134 + }, + { + "budget": 0.24, + "threshold": -0.09335668385028839, + "operator": "ge", + "actual_fraction": 0.23990260670295044, + "metric": 0.8678029217989115, + "rescued": 553, + "harmed": 67, + "missed_rescue": 601, + "avoided_harm": 341, + "selected_both_correct": 916, + "selected_both_wrong": 139 + }, + { + "budget": 0.25, + "threshold": -0.1357729732990265, + "operator": "ge", + "actual_fraction": 0.2499283872815812, + "metric": 0.8699513033514752, + "rescued": 574, + "harmed": 73, + "missed_rescue": 580, + "avoided_harm": 335, + "selected_both_correct": 956, + "selected_both_wrong": 142 + }, + { + "budget": 0.26, + "threshold": -0.19815389811992645, + "operator": "ge", + "actual_fraction": 0.25995416786021197, + "metric": 0.8708106559725007, + "rescued": 587, + "harmed": 80, + "missed_rescue": 567, + "avoided_harm": 328, + "selected_both_correct": 1005, + "selected_both_wrong": 143 + }, + { + "budget": 0.27, + "threshold": -0.2642267942428589, + "operator": "ge", + "actual_fraction": 0.26997994843884277, + "metric": 0.8723861357777141, + "rescued": 601, + "harmed": 83, + "missed_rescue": 553, + "avoided_harm": 325, + "selected_both_correct": 1056, + "selected_both_wrong": 145 + }, + { + "budget": 0.28, + "threshold": -0.33028537034988403, + "operator": "ge", + "actual_fraction": 0.2798625035806359, + "metric": 0.8736751647092523, + "rescued": 614, + "harmed": 87, + "missed_rescue": 540, + "avoided_harm": 321, + "selected_both_correct": 1104, + "selected_both_wrong": 149 + }, + { + "budget": 0.29, + "threshold": -0.39661598205566406, + "operator": "ge", + "actual_fraction": 0.2898882841592667, + "metric": 0.8756803208249785, + "rescued": 631, + "harmed": 90, + "missed_rescue": 523, + "avoided_harm": 318, + "selected_both_correct": 1152, + "selected_both_wrong": 151 + }, + { + "budget": 0.3, + "threshold": -0.47074592113494873, + "operator": "ge", + "actual_fraction": 0.29991406473789745, + "metric": 0.878258378688055, + "rescued": 650, + "harmed": 91, + "missed_rescue": 504, + "avoided_harm": 317, + "selected_both_correct": 1199, + "selected_both_wrong": 154 + }, + { + "budget": 0.31, + "threshold": -0.5337613821029663, + "operator": "ge", + "actual_fraction": 0.30993984531652824, + "metric": 0.8796906330564308, + "rescued": 664, + "harmed": 95, + "missed_rescue": 490, + "avoided_harm": 313, + "selected_both_correct": 1249, + "selected_both_wrong": 156 + }, + { + "budget": 0.32, + "threshold": -0.6040171384811401, + "operator": "ge", + "actual_fraction": 0.319965625895159, + "metric": 0.8801203093669435, + "rescued": 672, + "harmed": 100, + "missed_rescue": 482, + "avoided_harm": 308, + "selected_both_correct": 1302, + "selected_both_wrong": 160 + }, + { + "budget": 0.33, + "threshold": -0.6651467084884644, + "operator": "ge", + "actual_fraction": 0.3299914064737898, + "metric": 0.8814093382984818, + "rescued": 685, + "harmed": 104, + "missed_rescue": 469, + "avoided_harm": 304, + "selected_both_correct": 1353, + "selected_both_wrong": 162 + }, + { + "budget": 0.34, + "threshold": -0.72292160987854, + "operator": "ge", + "actual_fraction": 0.3398739616155829, + "metric": 0.8818390146089946, + "rescued": 691, + "harmed": 107, + "missed_rescue": 463, + "avoided_harm": 301, + "selected_both_correct": 1412, + "selected_both_wrong": 163 + }, + { + "budget": 0.35, + "threshold": -0.7783006429672241, + "operator": "ge", + "actual_fraction": 0.3498997421942137, + "metric": 0.8837009452878831, + "rescued": 707, + "harmed": 110, + "missed_rescue": 447, + "avoided_harm": 298, + "selected_both_correct": 1456, + "selected_both_wrong": 170 + }, + { + "budget": 0.36, + "threshold": -0.8395414352416992, + "operator": "ge", + "actual_fraction": 0.35992552277284445, + "metric": 0.8845602979089087, + "rescued": 716, + "harmed": 113, + "missed_rescue": 438, + "avoided_harm": 295, + "selected_both_correct": 1512, + "selected_both_wrong": 172 + }, + { + "budget": 0.37, + "threshold": -0.8945193886756897, + "operator": "ge", + "actual_fraction": 0.36995130335147525, + "metric": 0.8857061014036093, + "rescued": 726, + "harmed": 115, + "missed_rescue": 428, + "avoided_harm": 293, + "selected_both_correct": 1568, + "selected_both_wrong": 174 + }, + { + "budget": 0.38, + "threshold": -0.9500939249992371, + "operator": "ge", + "actual_fraction": 0.379977083930106, + "metric": 0.8868519048983099, + "rescued": 739, + "harmed": 120, + "missed_rescue": 415, + "avoided_harm": 288, + "selected_both_correct": 1618, + "selected_both_wrong": 176 + }, + { + "budget": 0.39, + "threshold": -0.9984015226364136, + "operator": "ge", + "actual_fraction": 0.3898596390718992, + "metric": 0.887854482956173, + "rescued": 750, + "harmed": 124, + "missed_rescue": 404, + "avoided_harm": 284, + "selected_both_correct": 1672, + "selected_both_wrong": 176 + }, + { + "budget": 0.4, + "threshold": -1.0433366298675537, + "operator": "ge", + "actual_fraction": 0.3998854196505299, + "metric": 0.8890002864508737, + "rescued": 762, + "harmed": 128, + "missed_rescue": 392, + "avoided_harm": 280, + "selected_both_correct": 1723, + "selected_both_wrong": 179 + }, + { + "budget": 0.41, + "threshold": -1.0980416536331177, + "operator": "ge", + "actual_fraction": 0.4099112002291607, + "metric": 0.889573188198224, + "rescued": 770, + "harmed": 132, + "missed_rescue": 384, + "avoided_harm": 276, + "selected_both_correct": 1779, + "selected_both_wrong": 181 + }, + { + "budget": 0.42, + "threshold": -1.1449317932128906, + "operator": "ge", + "actual_fraction": 0.41993698080779146, + "metric": 0.890575766256087, + "rescued": 781, + "harmed": 136, + "missed_rescue": 373, + "avoided_harm": 272, + "selected_both_correct": 1829, + "selected_both_wrong": 186 + }, + { + "budget": 0.43, + "threshold": -1.1991312503814697, + "operator": "ge", + "actual_fraction": 0.42996276138642225, + "metric": 0.8917215697507878, + "rescued": 793, + "harmed": 140, + "missed_rescue": 361, + "avoided_harm": 268, + "selected_both_correct": 1881, + "selected_both_wrong": 188 + }, + { + "budget": 0.44, + "threshold": -1.252983570098877, + "operator": "ge", + "actual_fraction": 0.439988541965053, + "metric": 0.893010598682326, + "rescued": 804, + "harmed": 142, + "missed_rescue": 350, + "avoided_harm": 266, + "selected_both_correct": 1936, + "selected_both_wrong": 190 + }, + { + "budget": 0.45, + "threshold": -1.3137803077697754, + "operator": "ge", + "actual_fraction": 0.4498710971068462, + "metric": 0.8934402749928387, + "rescued": 812, + "harmed": 147, + "missed_rescue": 342, + "avoided_harm": 261, + "selected_both_correct": 1989, + "selected_both_wrong": 193 + }, + { + "budget": 0.46, + "threshold": -1.3615622520446777, + "operator": "ge", + "actual_fraction": 0.45989687768547693, + "metric": 0.8931538241191636, + "rescued": 816, + "harmed": 153, + "missed_rescue": 338, + "avoided_harm": 255, + "selected_both_correct": 2048, + "selected_both_wrong": 194 + }, + { + "budget": 0.47, + "threshold": -1.4194676876068115, + "operator": "ge", + "actual_fraction": 0.46992265826410773, + "metric": 0.8941564021770266, + "rescued": 826, + "harmed": 156, + "missed_rescue": 328, + "avoided_harm": 252, + "selected_both_correct": 2101, + "selected_both_wrong": 198 + }, + { + "budget": 0.48, + "threshold": -1.4817214012145996, + "operator": "ge", + "actual_fraction": 0.47994843884273847, + "metric": 0.8950157547980522, + "rescued": 838, + "harmed": 162, + "missed_rescue": 316, + "avoided_harm": 246, + "selected_both_correct": 2153, + "selected_both_wrong": 198 + }, + { + "budget": 0.49, + "threshold": -1.52247953414917, + "operator": "ge", + "actual_fraction": 0.48997421942136926, + "metric": 0.8957318819822401, + "rescued": 847, + "harmed": 166, + "missed_rescue": 307, + "avoided_harm": 242, + "selected_both_correct": 2208, + "selected_both_wrong": 200 + }, + { + "budget": 0.5, + "threshold": -1.5746532678604126, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.8957318819822401, + "rescued": 852, + "harmed": 171, + "missed_rescue": 302, + "avoided_harm": 237, + "selected_both_correct": 2264, + "selected_both_wrong": 204 + }, + { + "budget": 0.51, + "threshold": -1.622488260269165, + "operator": "ge", + "actual_fraction": 0.5098825551417931, + "metric": 0.8967344600401032, + "rescued": 863, + "harmed": 175, + "missed_rescue": 291, + "avoided_harm": 233, + "selected_both_correct": 2317, + "selected_both_wrong": 205 + }, + { + "budget": 0.52, + "threshold": -1.66807222366333, + "operator": "ge", + "actual_fraction": 0.5199083357204239, + "metric": 0.8968776854769407, + "rescued": 869, + "harmed": 180, + "missed_rescue": 285, + "avoided_harm": 228, + "selected_both_correct": 2376, + "selected_both_wrong": 205 + }, + { + "budget": 0.53, + "threshold": -1.7201745510101318, + "operator": "ge", + "actual_fraction": 0.5299341162990547, + "metric": 0.8980234889716414, + "rescued": 881, + "harmed": 184, + "missed_rescue": 273, + "avoided_harm": 224, + "selected_both_correct": 2427, + "selected_both_wrong": 208 + }, + { + "budget": 0.54, + "threshold": -1.7738821506500244, + "operator": "ge", + "actual_fraction": 0.5399598968776855, + "metric": 0.8993125179031796, + "rescued": 890, + "harmed": 184, + "missed_rescue": 264, + "avoided_harm": 224, + "selected_both_correct": 2486, + "selected_both_wrong": 210 + }, + { + "budget": 0.55, + "threshold": -1.8242194652557373, + "operator": "ge", + "actual_fraction": 0.5499856774563162, + "metric": 0.8990260670295045, + "rescued": 893, + "harmed": 189, + "missed_rescue": 261, + "avoided_harm": 219, + "selected_both_correct": 2547, + "selected_both_wrong": 211 + }, + { + "budget": 0.56, + "threshold": -1.8723454475402832, + "operator": "ge", + "actual_fraction": 0.5598682325981095, + "metric": 0.8995989687768547, + "rescued": 902, + "harmed": 194, + "missed_rescue": 252, + "avoided_harm": 214, + "selected_both_correct": 2599, + "selected_both_wrong": 214 + }, + { + "budget": 0.57, + "threshold": -1.9198856353759766, + "operator": "ge", + "actual_fraction": 0.5698940131767402, + "metric": 0.9001718705242051, + "rescued": 911, + "harmed": 199, + "missed_rescue": 243, + "avoided_harm": 209, + "selected_both_correct": 2652, + "selected_both_wrong": 217 + }, + { + "budget": 0.58, + "threshold": -1.9657607078552246, + "operator": "ge", + "actual_fraction": 0.579919793755371, + "metric": 0.9010312231452305, + "rescued": 923, + "harmed": 205, + "missed_rescue": 231, + "avoided_harm": 203, + "selected_both_correct": 2703, + "selected_both_wrong": 218 + }, + { + "budget": 0.59, + "threshold": -2.008908748626709, + "operator": "ge", + "actual_fraction": 0.5899455743340017, + "metric": 0.9021770266399313, + "rescued": 935, + "harmed": 209, + "missed_rescue": 219, + "avoided_harm": 199, + "selected_both_correct": 2756, + "selected_both_wrong": 219 + }, + { + "budget": 0.6, + "threshold": -2.0556583404541016, + "operator": "ge", + "actual_fraction": 0.5999713549126325, + "metric": 0.9027499283872816, + "rescued": 943, + "harmed": 213, + "missed_rescue": 211, + "avoided_harm": 195, + "selected_both_correct": 2814, + "selected_both_wrong": 219 + }, + { + "budget": 0.61, + "threshold": -2.1030964851379395, + "operator": "ge", + "actual_fraction": 0.6099971354912632, + "metric": 0.902606702950444, + "rescued": 946, + "harmed": 217, + "missed_rescue": 208, + "avoided_harm": 191, + "selected_both_correct": 2876, + "selected_both_wrong": 220 + }, + { + "budget": 0.62, + "threshold": -2.156914710998535, + "operator": "ge", + "actual_fraction": 0.6198796906330565, + "metric": 0.9030363792609567, + "rescued": 952, + "harmed": 220, + "missed_rescue": 202, + "avoided_harm": 188, + "selected_both_correct": 2936, + "selected_both_wrong": 220 + }, + { + "budget": 0.63, + "threshold": -2.2176589965820312, + "operator": "ge", + "actual_fraction": 0.6299054712116872, + "metric": 0.9027499283872816, + "rescued": 961, + "harmed": 231, + "missed_rescue": 193, + "avoided_harm": 177, + "selected_both_correct": 2986, + "selected_both_wrong": 220 + }, + { + "budget": 0.64, + "threshold": -2.2867987155914307, + "operator": "ge", + "actual_fraction": 0.639931251790318, + "metric": 0.9027499283872816, + "rescued": 966, + "harmed": 236, + "missed_rescue": 188, + "avoided_harm": 172, + "selected_both_correct": 3044, + "selected_both_wrong": 222 + }, + { + "budget": 0.65, + "threshold": -2.3520939350128174, + "operator": "ge", + "actual_fraction": 0.6499570323689488, + "metric": 0.9038957318819822, + "rescued": 978, + "harmed": 240, + "missed_rescue": 176, + "avoided_harm": 168, + "selected_both_correct": 3095, + "selected_both_wrong": 225 + }, + { + "budget": 0.66, + "threshold": -2.4134349822998047, + "operator": "ge", + "actual_fraction": 0.6599828129475795, + "metric": 0.9046118590661701, + "rescued": 988, + "harmed": 245, + "missed_rescue": 166, + "avoided_harm": 163, + "selected_both_correct": 3150, + "selected_both_wrong": 225 + }, + { + "budget": 0.67, + "threshold": -2.4683873653411865, + "operator": "ge", + "actual_fraction": 0.6698653680893727, + "metric": 0.9051847608135205, + "rescued": 997, + "harmed": 250, + "missed_rescue": 157, + "avoided_harm": 158, + "selected_both_correct": 3204, + "selected_both_wrong": 226 + }, + { + "budget": 0.68, + "threshold": -2.5484306812286377, + "operator": "ge", + "actual_fraction": 0.6798911486680035, + "metric": 0.905327986250358, + "rescued": 1002, + "harmed": 254, + "missed_rescue": 152, + "avoided_harm": 154, + "selected_both_correct": 3265, + "selected_both_wrong": 226 + }, + { + "budget": 0.69, + "threshold": -2.608332633972168, + "operator": "ge", + "actual_fraction": 0.6899169292466342, + "metric": 0.9054712116871957, + "rescued": 1007, + "harmed": 258, + "missed_rescue": 147, + "avoided_harm": 150, + "selected_both_correct": 3324, + "selected_both_wrong": 228 + }, + { + "budget": 0.7, + "threshold": -2.670351505279541, + "operator": "ge", + "actual_fraction": 0.699942709825265, + "metric": 0.9060441134345459, + "rescued": 1014, + "harmed": 261, + "missed_rescue": 140, + "avoided_harm": 147, + "selected_both_correct": 3381, + "selected_both_wrong": 231 + }, + { + "budget": 0.71, + "threshold": -2.745088577270508, + "operator": "ge", + "actual_fraction": 0.7099684904038958, + "metric": 0.9060441134345459, + "rescued": 1018, + "harmed": 265, + "missed_rescue": 136, + "avoided_harm": 143, + "selected_both_correct": 3442, + "selected_both_wrong": 232 + }, + { + "budget": 0.72, + "threshold": -2.8100666999816895, + "operator": "ge", + "actual_fraction": 0.7199942709825266, + "metric": 0.9057576625608708, + "rescued": 1022, + "harmed": 271, + "missed_rescue": 132, + "avoided_harm": 137, + "selected_both_correct": 3499, + "selected_both_wrong": 235 + }, + { + "budget": 0.73, + "threshold": -2.867399215698242, + "operator": "ge", + "actual_fraction": 0.7298768261243197, + "metric": 0.9067602406187338, + "rescued": 1033, + "harmed": 275, + "missed_rescue": 121, + "avoided_harm": 133, + "selected_both_correct": 3552, + "selected_both_wrong": 236 + }, + { + "budget": 0.74, + "threshold": -2.953387975692749, + "operator": "ge", + "actual_fraction": 0.7399026067029505, + "metric": 0.9077628186765969, + "rescued": 1042, + "harmed": 277, + "missed_rescue": 112, + "avoided_harm": 131, + "selected_both_correct": 3608, + "selected_both_wrong": 239 + }, + { + "budget": 0.75, + "threshold": -3.0269675254821777, + "operator": "ge", + "actual_fraction": 0.7499283872815812, + "metric": 0.9080492695502721, + "rescued": 1049, + "harmed": 282, + "missed_rescue": 105, + "avoided_harm": 126, + "selected_both_correct": 3666, + "selected_both_wrong": 239 + }, + { + "budget": 0.76, + "threshold": -3.0986766815185547, + "operator": "ge", + "actual_fraction": 0.759954167860212, + "metric": 0.90876539673446, + "rescued": 1058, + "harmed": 286, + "missed_rescue": 96, + "avoided_harm": 122, + "selected_both_correct": 3723, + "selected_both_wrong": 239 + }, + { + "budget": 0.77, + "threshold": -3.183574676513672, + "operator": "ge", + "actual_fraction": 0.7699799484388428, + "metric": 0.909481523918648, + "rescued": 1066, + "harmed": 289, + "missed_rescue": 88, + "avoided_harm": 119, + "selected_both_correct": 3781, + "selected_both_wrong": 240 + }, + { + "budget": 0.78, + "threshold": -3.2600371837615967, + "operator": "ge", + "actual_fraction": 0.7798625035806359, + "metric": 0.9093382984818104, + "rescued": 1070, + "harmed": 294, + "missed_rescue": 84, + "avoided_harm": 114, + "selected_both_correct": 3839, + "selected_both_wrong": 242 + }, + { + "budget": 0.79, + "threshold": -3.3452396392822266, + "operator": "ge", + "actual_fraction": 0.7898882841592667, + "metric": 0.9091950730449728, + "rescued": 1075, + "harmed": 300, + "missed_rescue": 79, + "avoided_harm": 108, + "selected_both_correct": 3895, + "selected_both_wrong": 245 + }, + { + "budget": 0.8, + "threshold": -3.4057769775390625, + "operator": "ge", + "actual_fraction": 0.7999140647378975, + "metric": 0.9090518476081352, + "rescued": 1082, + "harmed": 308, + "missed_rescue": 72, + "avoided_harm": 100, + "selected_both_correct": 3950, + "selected_both_wrong": 245 + }, + { + "budget": 0.81, + "threshold": -3.474921464920044, + "operator": "ge", + "actual_fraction": 0.8099398453165282, + "metric": 0.9090518476081352, + "rescued": 1085, + "harmed": 311, + "missed_rescue": 69, + "avoided_harm": 97, + "selected_both_correct": 4014, + "selected_both_wrong": 245 + }, + { + "budget": 0.82, + "threshold": -3.5584769248962402, + "operator": "ge", + "actual_fraction": 0.819965625895159, + "metric": 0.9086221712976225, + "rescued": 1088, + "harmed": 317, + "missed_rescue": 66, + "avoided_harm": 91, + "selected_both_correct": 4075, + "selected_both_wrong": 245 + }, + { + "budget": 0.83, + "threshold": -3.660430431365967, + "operator": "ge", + "actual_fraction": 0.8299914064737898, + "metric": 0.9090518476081352, + "rescued": 1095, + "harmed": 321, + "missed_rescue": 59, + "avoided_harm": 87, + "selected_both_correct": 4133, + "selected_both_wrong": 246 + }, + { + "budget": 0.84, + "threshold": -3.760495185852051, + "operator": "ge", + "actual_fraction": 0.8398739616155829, + "metric": 0.9086221712976225, + "rescued": 1098, + "harmed": 327, + "missed_rescue": 56, + "avoided_harm": 81, + "selected_both_correct": 4192, + "selected_both_wrong": 247 + }, + { + "budget": 0.85, + "threshold": -3.8480324745178223, + "operator": "ge", + "actual_fraction": 0.8498997421942137, + "metric": 0.9086221712976225, + "rescued": 1101, + "harmed": 330, + "missed_rescue": 53, + "avoided_harm": 78, + "selected_both_correct": 4255, + "selected_both_wrong": 248 + }, + { + "budget": 0.86, + "threshold": -3.9325687885284424, + "operator": "ge", + "actual_fraction": 0.8599255227728445, + "metric": 0.9093382984818104, + "rescued": 1109, + "harmed": 333, + "missed_rescue": 45, + "avoided_harm": 75, + "selected_both_correct": 4314, + "selected_both_wrong": 248 + }, + { + "budget": 0.87, + "threshold": -4.00416374206543, + "operator": "ge", + "actual_fraction": 0.8699513033514752, + "metric": 0.9089086221712976, + "rescued": 1110, + "harmed": 337, + "missed_rescue": 44, + "avoided_harm": 71, + "selected_both_correct": 4379, + "selected_both_wrong": 248 + }, + { + "budget": 0.88, + "threshold": -4.082941055297852, + "operator": "ge", + "actual_fraction": 0.879977083930106, + "metric": 0.9089086221712976, + "rescued": 1113, + "harmed": 340, + "missed_rescue": 41, + "avoided_harm": 68, + "selected_both_correct": 4443, + "selected_both_wrong": 248 + }, + { + "budget": 0.89, + "threshold": -4.1705641746521, + "operator": "ge", + "actual_fraction": 0.8898596390718991, + "metric": 0.9091950730449728, + "rescued": 1120, + "harmed": 345, + "missed_rescue": 34, + "avoided_harm": 63, + "selected_both_correct": 4500, + "selected_both_wrong": 248 + }, + { + "budget": 0.9, + "threshold": -4.255984306335449, + "operator": "ge", + "actual_fraction": 0.8998854196505299, + "metric": 0.9090518476081352, + "rescued": 1122, + "harmed": 348, + "missed_rescue": 32, + "avoided_harm": 60, + "selected_both_correct": 4564, + "selected_both_wrong": 249 + }, + { + "budget": 0.91, + "threshold": -4.356119155883789, + "operator": "ge", + "actual_fraction": 0.9099112002291607, + "metric": 0.9089086221712976, + "rescued": 1129, + "harmed": 356, + "missed_rescue": 25, + "avoided_harm": 52, + "selected_both_correct": 4618, + "selected_both_wrong": 250 + }, + { + "budget": 0.92, + "threshold": -4.483384132385254, + "operator": "ge", + "actual_fraction": 0.9199369808077915, + "metric": 0.90876539673446, + "rescued": 1132, + "harmed": 360, + "missed_rescue": 22, + "avoided_harm": 48, + "selected_both_correct": 4680, + "selected_both_wrong": 251 + }, + { + "budget": 0.93, + "threshold": -4.5789947509765625, + "operator": "ge", + "actual_fraction": 0.9299627613864222, + "metric": 0.90876539673446, + "rescued": 1135, + "harmed": 363, + "missed_rescue": 19, + "avoided_harm": 45, + "selected_both_correct": 4744, + "selected_both_wrong": 251 + }, + { + "budget": 0.94, + "threshold": -4.691610336303711, + "operator": "ge", + "actual_fraction": 0.939988541965053, + "metric": 0.9086221712976225, + "rescued": 1140, + "harmed": 369, + "missed_rescue": 14, + "avoided_harm": 39, + "selected_both_correct": 4803, + "selected_both_wrong": 251 + }, + { + "budget": 0.95, + "threshold": -4.819090843200684, + "operator": "ge", + "actual_fraction": 0.9498710971068461, + "metric": 0.9083357204239473, + "rescued": 1143, + "harmed": 374, + "missed_rescue": 11, + "avoided_harm": 34, + "selected_both_correct": 4864, + "selected_both_wrong": 251 + }, + { + "budget": 0.96, + "threshold": -4.976077079772949, + "operator": "ge", + "actual_fraction": 0.9598968776854769, + "metric": 0.9079060441134346, + "rescued": 1146, + "harmed": 380, + "missed_rescue": 8, + "avoided_harm": 28, + "selected_both_correct": 4925, + "selected_both_wrong": 251 + }, + { + "budget": 0.97, + "threshold": -5.13118839263916, + "operator": "ge", + "actual_fraction": 0.9699226582641077, + "metric": 0.9077628186765969, + "rescued": 1151, + "harmed": 386, + "missed_rescue": 3, + "avoided_harm": 22, + "selected_both_correct": 4983, + "selected_both_wrong": 252 + }, + { + "budget": 0.98, + "threshold": -5.317582130432129, + "operator": "ge", + "actual_fraction": 0.9799484388427385, + "metric": 0.9071899169292467, + "rescued": 1153, + "harmed": 392, + "missed_rescue": 1, + "avoided_harm": 16, + "selected_both_correct": 5043, + "selected_both_wrong": 254 + }, + { + "budget": 0.99, + "threshold": -5.640842437744141, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.9069034660555715, + "rescued": 1154, + "harmed": 395, + "missed_rescue": 0, + "avoided_harm": 13, + "selected_both_correct": 5109, + "selected_both_wrong": 254 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9050415353766829, + "rescued": 1154, + "harmed": 408, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 5165, + "selected_both_wrong": 255 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.8000287005811868, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2247, + "avoided_harm": 845, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.821803569793701, + "operator": "ge", + "actual_fraction": 0.011264978115806845, + "metric": 0.8065580828011767, + "rescued": 93, + "harmed": 2, + "missed_rescue": 2154, + "avoided_harm": 843, + "selected_both_correct": 54, + "selected_both_wrong": 8 + }, + { + "budget": 0.02, + "threshold": 2.459282875061035, + "operator": "ge", + "actual_fraction": 0.02116667862524216, + "metric": 0.8116524359618282, + "rescued": 165, + "harmed": 3, + "missed_rescue": 2082, + "avoided_harm": 842, + "selected_both_correct": 109, + "selected_both_wrong": 18 + }, + { + "budget": 0.03, + "threshold": 2.2472119331359863, + "operator": "ge", + "actual_fraction": 0.032718662552916694, + "metric": 0.8163880318576451, + "rescued": 240, + "harmed": 12, + "missed_rescue": 2007, + "avoided_harm": 833, + "selected_both_correct": 173, + "selected_both_wrong": 31 + }, + { + "budget": 0.04, + "threshold": 2.0771026611328125, + "operator": "ge", + "actual_fraction": 0.04348138049795509, + "metric": 0.8208366219415943, + "rescued": 307, + "harmed": 17, + "missed_rescue": 1940, + "avoided_harm": 828, + "selected_both_correct": 239, + "selected_both_wrong": 43 + }, + { + "budget": 0.05, + "threshold": 1.8918991088867188, + "operator": "ge", + "actual_fraction": 0.055822630408265766, + "metric": 0.8252134605725766, + "rescued": 375, + "harmed": 24, + "missed_rescue": 1872, + "avoided_harm": 821, + "selected_both_correct": 318, + "selected_both_wrong": 61 + }, + { + "budget": 0.06, + "threshold": 1.754787564277649, + "operator": "ge", + "actual_fraction": 0.06471981057616416, + "metric": 0.8289445361268566, + "rescued": 430, + "harmed": 27, + "missed_rescue": 1817, + "avoided_harm": 818, + "selected_both_correct": 373, + "selected_both_wrong": 72 + }, + { + "budget": 0.07, + "threshold": 1.612305998802185, + "operator": "ge", + "actual_fraction": 0.07533902561526871, + "metric": 0.8325321087752027, + "rescued": 485, + "harmed": 32, + "missed_rescue": 1762, + "avoided_harm": 813, + "selected_both_correct": 444, + "selected_both_wrong": 89 + }, + { + "budget": 0.08, + "threshold": 1.4815378189086914, + "operator": "ge", + "actual_fraction": 0.08466671450096865, + "metric": 0.8348281552701442, + "rescued": 521, + "harmed": 36, + "missed_rescue": 1726, + "avoided_harm": 809, + "selected_both_correct": 515, + "selected_both_wrong": 108 + }, + { + "budget": 0.09, + "threshold": 1.3550769090652466, + "operator": "ge", + "actual_fraction": 0.09370739757480089, + "metric": 0.8375547104828873, + "rescued": 565, + "harmed": 42, + "missed_rescue": 1682, + "avoided_harm": 803, + "selected_both_correct": 579, + "selected_both_wrong": 120 + }, + { + "budget": 0.1, + "threshold": 1.2258758544921875, + "operator": "ge", + "actual_fraction": 0.10317858936643468, + "metric": 0.839635502618928, + "rescued": 598, + "harmed": 46, + "missed_rescue": 1649, + "avoided_harm": 799, + "selected_both_correct": 657, + "selected_both_wrong": 137 + }, + { + "budget": 0.11, + "threshold": 1.0788583755493164, + "operator": "ge", + "actual_fraction": 0.1134390471407046, + "metric": 0.8424338092846381, + "rescued": 640, + "harmed": 49, + "missed_rescue": 1607, + "avoided_harm": 796, + "selected_both_correct": 736, + "selected_both_wrong": 156 + }, + { + "budget": 0.12, + "threshold": 0.9625301957130432, + "operator": "ge", + "actual_fraction": 0.12219272440266915, + "metric": 0.8443710985147449, + "rescued": 676, + "harmed": 58, + "missed_rescue": 1571, + "avoided_harm": 787, + "selected_both_correct": 796, + "selected_both_wrong": 173 + }, + { + "budget": 0.13, + "threshold": 0.8249785304069519, + "operator": "ge", + "actual_fraction": 0.133457702518476, + "metric": 0.848030422616058, + "rescued": 731, + "harmed": 62, + "missed_rescue": 1516, + "avoided_harm": 783, + "selected_both_correct": 881, + "selected_both_wrong": 186 + }, + { + "budget": 0.14, + "threshold": 0.7093843221664429, + "operator": "ge", + "actual_fraction": 0.14343115448087823, + "metric": 0.8503982205639664, + "rescued": 767, + "harmed": 65, + "missed_rescue": 1480, + "avoided_harm": 780, + "selected_both_correct": 967, + "selected_both_wrong": 200 + }, + { + "budget": 0.15, + "threshold": 0.5911833047866821, + "operator": "ge", + "actual_fraction": 0.15534189567338738, + "metric": 0.8532682786826433, + "rescued": 813, + "harmed": 71, + "missed_rescue": 1434, + "avoided_harm": 774, + "selected_both_correct": 1069, + "selected_both_wrong": 212 + }, + { + "budget": 0.16, + "threshold": 0.5016208291053772, + "operator": "ge", + "actual_fraction": 0.16574585635359115, + "metric": 0.8560665853483533, + "rescued": 853, + "harmed": 72, + "missed_rescue": 1394, + "avoided_harm": 773, + "selected_both_correct": 1162, + "selected_both_wrong": 223 + }, + { + "budget": 0.17, + "threshold": 0.44066739082336426, + "operator": "ge", + "actual_fraction": 0.17270574729138266, + "metric": 0.8579321231254933, + "rescued": 882, + "harmed": 75, + "missed_rescue": 1365, + "avoided_harm": 770, + "selected_both_correct": 1223, + "selected_both_wrong": 227 + }, + { + "budget": 0.18, + "threshold": 0.37089839577674866, + "operator": "ge", + "actual_fraction": 0.18117241874147952, + "metric": 0.8597976609026333, + "rescued": 913, + "harmed": 80, + "missed_rescue": 1334, + "avoided_harm": 765, + "selected_both_correct": 1297, + "selected_both_wrong": 235 + }, + { + "budget": 0.19, + "threshold": 0.2769058346748352, + "operator": "ge", + "actual_fraction": 0.19337016574585636, + "metric": 0.8625242161153763, + "rescued": 958, + "harmed": 87, + "missed_rescue": 1289, + "avoided_harm": 758, + "selected_both_correct": 1405, + "selected_both_wrong": 245 + }, + { + "budget": 0.2, + "threshold": 0.20165908336639404, + "operator": "ge", + "actual_fraction": 0.20312836334935783, + "metric": 0.863815742268781, + "rescued": 983, + "harmed": 94, + "missed_rescue": 1264, + "avoided_harm": 751, + "selected_both_correct": 1497, + "selected_both_wrong": 257 + }, + { + "budget": 0.21, + "threshold": 0.13794422149658203, + "operator": "ge", + "actual_fraction": 0.21001650283418238, + "metric": 0.8652507713281193, + "rescued": 1008, + "harmed": 99, + "missed_rescue": 1239, + "avoided_harm": 746, + "selected_both_correct": 1556, + "selected_both_wrong": 264 + }, + { + "budget": 0.22, + "threshold": 0.06396147608757019, + "operator": "ge", + "actual_fraction": 0.22034871206141923, + "metric": 0.8663270431226232, + "rescued": 1034, + "harmed": 110, + "missed_rescue": 1213, + "avoided_harm": 735, + "selected_both_correct": 1651, + "selected_both_wrong": 276 + }, + { + "budget": 0.23, + "threshold": -0.017305493354797363, + "operator": "ge", + "actual_fraction": 0.2323312047068953, + "metric": 0.869340604147234, + "rescued": 1083, + "harmed": 117, + "missed_rescue": 1164, + "avoided_harm": 728, + "selected_both_correct": 1749, + "selected_both_wrong": 289 + }, + { + "budget": 0.24, + "threshold": -0.09335668385028839, + "operator": "ge", + "actual_fraction": 0.244672454617206, + "metric": 0.8714931477362416, + "rescued": 1120, + "harmed": 124, + "missed_rescue": 1127, + "avoided_harm": 721, + "selected_both_correct": 1867, + "selected_both_wrong": 299 + }, + { + "budget": 0.25, + "threshold": -0.1357729732990265, + "operator": "ge", + "actual_fraction": 0.24926454760708905, + "metric": 0.8724259166248116, + "rescued": 1136, + "harmed": 127, + "missed_rescue": 1111, + "avoided_harm": 718, + "selected_both_correct": 1909, + "selected_both_wrong": 302 + }, + { + "budget": 0.26, + "threshold": -0.19815389811992645, + "operator": "ge", + "actual_fraction": 0.25909449666355744, + "metric": 0.8737174427782163, + "rescued": 1164, + "harmed": 137, + "missed_rescue": 1083, + "avoided_harm": 708, + "selected_both_correct": 2002, + "selected_both_wrong": 308 + }, + { + "budget": 0.27, + "threshold": -0.2642267942428589, + "operator": "ge", + "actual_fraction": 0.26734591375475353, + "metric": 0.874865466025687, + "rescued": 1186, + "harmed": 143, + "missed_rescue": 1061, + "avoided_harm": 702, + "selected_both_correct": 2080, + "selected_both_wrong": 317 + }, + { + "budget": 0.28, + "threshold": -0.33028537034988403, + "operator": "ge", + "actual_fraction": 0.27545382794001577, + "metric": 0.8765157494439262, + "rescued": 1215, + "harmed": 149, + "missed_rescue": 1032, + "avoided_harm": 696, + "selected_both_correct": 2156, + "selected_both_wrong": 319 + }, + { + "budget": 0.29, + "threshold": -0.39661598205566406, + "operator": "ge", + "actual_fraction": 0.28399225084307955, + "metric": 0.8782377843151323, + "rescued": 1248, + "harmed": 158, + "missed_rescue": 999, + "avoided_harm": 687, + "selected_both_correct": 2229, + "selected_both_wrong": 323 + }, + { + "budget": 0.3, + "threshold": -0.47074592113494873, + "operator": "ge", + "actual_fraction": 0.29669225801822485, + "metric": 0.8798880677333716, + "rescued": 1283, + "harmed": 170, + "missed_rescue": 964, + "avoided_harm": 675, + "selected_both_correct": 2349, + "selected_both_wrong": 333 + }, + { + "budget": 0.31, + "threshold": -0.5337613821029663, + "operator": "ge", + "actual_fraction": 0.3078854846810648, + "metric": 0.8818253569634785, + "rescued": 1318, + "harmed": 178, + "missed_rescue": 929, + "avoided_harm": 667, + "selected_both_correct": 2457, + "selected_both_wrong": 338 + }, + { + "budget": 0.32, + "threshold": -0.6040171384811401, + "operator": "ge", + "actual_fraction": 0.3177871851905001, + "metric": 0.8829016287579824, + "rescued": 1339, + "harmed": 184, + "missed_rescue": 908, + "avoided_harm": 661, + "selected_both_correct": 2559, + "selected_both_wrong": 347 + }, + { + "budget": 0.33, + "threshold": -0.6651467084884644, + "operator": "ge", + "actual_fraction": 0.32840640022960466, + "metric": 0.8842649063643538, + "rescued": 1365, + "harmed": 191, + "missed_rescue": 882, + "avoided_harm": 654, + "selected_both_correct": 2667, + "selected_both_wrong": 354 + }, + { + "budget": 0.34, + "threshold": -0.72292160987854, + "operator": "ge", + "actual_fraction": 0.33981488125134535, + "metric": 0.8852694267058908, + "rescued": 1385, + "harmed": 197, + "missed_rescue": 862, + "avoided_harm": 648, + "selected_both_correct": 2793, + "selected_both_wrong": 361 + }, + { + "budget": 0.35, + "threshold": -0.7783006429672241, + "operator": "ge", + "actual_fraction": 0.3476357896247399, + "metric": 0.88598694123556, + "rescued": 1404, + "harmed": 206, + "missed_rescue": 843, + "avoided_harm": 639, + "selected_both_correct": 2871, + "selected_both_wrong": 364 + }, + { + "budget": 0.36, + "threshold": -0.8395414352416992, + "operator": "ge", + "actual_fraction": 0.35696347851043986, + "metric": 0.8871349644830308, + "rescued": 1426, + "harmed": 212, + "missed_rescue": 821, + "avoided_harm": 633, + "selected_both_correct": 2970, + "selected_both_wrong": 367 + }, + { + "budget": 0.37, + "threshold": -0.8945193886756897, + "operator": "ge", + "actual_fraction": 0.3658606586783382, + "metric": 0.8876372246537992, + "rescued": 1439, + "harmed": 218, + "missed_rescue": 808, + "avoided_harm": 627, + "selected_both_correct": 3070, + "selected_both_wrong": 372 + }, + { + "budget": 0.38, + "threshold": -0.9500939249992371, + "operator": "ge", + "actual_fraction": 0.3738250699576666, + "metric": 0.8879959819186338, + "rescued": 1455, + "harmed": 229, + "missed_rescue": 792, + "avoided_harm": 616, + "selected_both_correct": 3149, + "selected_both_wrong": 377 + }, + { + "budget": 0.39, + "threshold": -0.9984015226364136, + "operator": "ge", + "actual_fraction": 0.38250699576666425, + "metric": 0.8888569993542369, + "rescued": 1472, + "harmed": 234, + "missed_rescue": 775, + "avoided_harm": 611, + "selected_both_correct": 3244, + "selected_both_wrong": 381 + }, + { + "budget": 0.4, + "threshold": -1.0433366298675537, + "operator": "ge", + "actual_fraction": 0.3904714070459927, + "metric": 0.8894310109779723, + "rescued": 1486, + "harmed": 240, + "missed_rescue": 761, + "avoided_harm": 605, + "selected_both_correct": 3332, + "selected_both_wrong": 384 + }, + { + "budget": 0.41, + "threshold": -1.0980416536331177, + "operator": "ge", + "actual_fraction": 0.40015785319652725, + "metric": 0.8902920284135754, + "rescued": 1505, + "harmed": 247, + "missed_rescue": 742, + "avoided_harm": 598, + "selected_both_correct": 3434, + "selected_both_wrong": 391 + }, + { + "budget": 0.42, + "threshold": -1.1449317932128906, + "operator": "ge", + "actual_fraction": 0.4085527731936572, + "metric": 0.89158355456698, + "rescued": 1525, + "harmed": 249, + "missed_rescue": 722, + "avoided_harm": 596, + "selected_both_correct": 3522, + "selected_both_wrong": 398 + }, + { + "budget": 0.43, + "threshold": -1.1991312503814697, + "operator": "ge", + "actual_fraction": 0.41931549113869554, + "metric": 0.8919423118318146, + "rescued": 1542, + "harmed": 261, + "missed_rescue": 705, + "avoided_harm": 584, + "selected_both_correct": 3637, + "selected_both_wrong": 404 + }, + { + "budget": 0.44, + "threshold": -1.252983570098877, + "operator": "ge", + "actual_fraction": 0.43014996053670085, + "metric": 0.8936643467030208, + "rescued": 1569, + "harmed": 264, + "missed_rescue": 678, + "avoided_harm": 581, + "selected_both_correct": 3756, + "selected_both_wrong": 406 + }, + { + "budget": 0.45, + "threshold": -1.3137803077697754, + "operator": "ge", + "actual_fraction": 0.4423477075410777, + "metric": 0.8955298844801607, + "rescued": 1602, + "harmed": 271, + "missed_rescue": 645, + "avoided_harm": 574, + "selected_both_correct": 3880, + "selected_both_wrong": 412 + }, + { + "budget": 0.46, + "threshold": -1.3615622520446777, + "operator": "ge", + "actual_fraction": 0.45110138480304224, + "metric": 0.8961038961038961, + "rescued": 1616, + "harmed": 277, + "missed_rescue": 631, + "avoided_harm": 568, + "selected_both_correct": 3979, + "selected_both_wrong": 415 + }, + { + "budget": 0.47, + "threshold": -1.4194676876068115, + "operator": "ge", + "actual_fraction": 0.46129009112434527, + "metric": 0.8971801678983999, + "rescued": 1638, + "harmed": 284, + "missed_rescue": 609, + "avoided_harm": 561, + "selected_both_correct": 4085, + "selected_both_wrong": 422 + }, + { + "budget": 0.48, + "threshold": -1.4817214012145996, + "operator": "ge", + "actual_fraction": 0.47456410992322595, + "metric": 0.8972519193513668, + "rescued": 1657, + "harmed": 302, + "missed_rescue": 590, + "avoided_harm": 543, + "selected_both_correct": 4230, + "selected_both_wrong": 425 + }, + { + "budget": 0.49, + "threshold": -1.52247953414917, + "operator": "ge", + "actual_fraction": 0.4816675037669513, + "metric": 0.8976824280691684, + "rescued": 1668, + "harmed": 307, + "missed_rescue": 579, + "avoided_harm": 538, + "selected_both_correct": 4313, + "selected_both_wrong": 425 + }, + { + "budget": 0.5, + "threshold": -1.5746532678604126, + "operator": "ge", + "actual_fraction": 0.49199971299418815, + "metric": 0.8982564396929038, + "rescued": 1683, + "harmed": 314, + "missed_rescue": 564, + "avoided_harm": 531, + "selected_both_correct": 4432, + "selected_both_wrong": 428 + }, + { + "budget": 0.51, + "threshold": -1.622488260269165, + "operator": "ge", + "actual_fraction": 0.5021166678625242, + "metric": 0.8986151969577384, + "rescued": 1702, + "harmed": 328, + "missed_rescue": 545, + "avoided_harm": 517, + "selected_both_correct": 4535, + "selected_both_wrong": 433 + }, + { + "budget": 0.52, + "threshold": -1.66807222366333, + "operator": "ge", + "actual_fraction": 0.5105833393126211, + "metric": 0.8996914687522423, + "rescued": 1719, + "harmed": 330, + "missed_rescue": 528, + "avoided_harm": 515, + "selected_both_correct": 4630, + "selected_both_wrong": 437 + }, + { + "budget": 0.53, + "threshold": -1.7201745510101318, + "operator": "ge", + "actual_fraction": 0.5214178087106264, + "metric": 0.89997847456411, + "rescued": 1732, + "harmed": 339, + "missed_rescue": 515, + "avoided_harm": 506, + "selected_both_correct": 4757, + "selected_both_wrong": 439 + }, + { + "budget": 0.54, + "threshold": -1.7738821506500244, + "operator": "ge", + "actual_fraction": 0.5305302432374256, + "metric": 0.9001937289230106, + "rescued": 1742, + "harmed": 346, + "missed_rescue": 505, + "avoided_harm": 499, + "selected_both_correct": 4861, + "selected_both_wrong": 445 + }, + { + "budget": 0.55, + "threshold": -1.8242194652557373, + "operator": "ge", + "actual_fraction": 0.5414364640883977, + "metric": 0.9014135036234484, + "rescued": 1765, + "harmed": 352, + "missed_rescue": 482, + "avoided_harm": 493, + "selected_both_correct": 4981, + "selected_both_wrong": 448 + }, + { + "budget": 0.56, + "threshold": -1.8723454475402832, + "operator": "ge", + "actual_fraction": 0.5529166965631054, + "metric": 0.9024180239649853, + "rescued": 1787, + "harmed": 360, + "missed_rescue": 460, + "avoided_harm": 485, + "selected_both_correct": 5108, + "selected_both_wrong": 451 + }, + { + "budget": 0.57, + "threshold": -1.9198856353759766, + "operator": "ge", + "actual_fraction": 0.5631054028844084, + "metric": 0.9034225443065222, + "rescued": 1809, + "harmed": 368, + "missed_rescue": 438, + "avoided_harm": 477, + "selected_both_correct": 5217, + "selected_both_wrong": 454 + }, + { + "budget": 0.58, + "threshold": -1.9657607078552246, + "operator": "ge", + "actual_fraction": 0.5717155772404391, + "metric": 0.9042118102891583, + "rescued": 1825, + "harmed": 373, + "missed_rescue": 422, + "avoided_harm": 472, + "selected_both_correct": 5314, + "selected_both_wrong": 456 + }, + { + "budget": 0.59, + "threshold": -2.008908748626709, + "operator": "ge", + "actual_fraction": 0.5797517399727344, + "metric": 0.9042118102891583, + "rescued": 1835, + "harmed": 383, + "missed_rescue": 412, + "avoided_harm": 462, + "selected_both_correct": 5404, + "selected_both_wrong": 458 + }, + { + "budget": 0.6, + "threshold": -2.0556583404541016, + "operator": "ge", + "actual_fraction": 0.5881466599698644, + "metric": 0.9048575733658607, + "rescued": 1850, + "harmed": 389, + "missed_rescue": 397, + "avoided_harm": 456, + "selected_both_correct": 5500, + "selected_both_wrong": 458 + }, + { + "budget": 0.61, + "threshold": -2.1030964851379395, + "operator": "ge", + "actual_fraction": 0.5958240654373251, + "metric": 0.9052880820836622, + "rescued": 1863, + "harmed": 396, + "missed_rescue": 384, + "avoided_harm": 449, + "selected_both_correct": 5582, + "selected_both_wrong": 463 + }, + { + "budget": 0.62, + "threshold": -2.156914710998535, + "operator": "ge", + "actual_fraction": 0.6056540144937935, + "metric": 0.9065078567840998, + "rescued": 1886, + "harmed": 402, + "missed_rescue": 361, + "avoided_harm": 443, + "selected_both_correct": 5690, + "selected_both_wrong": 463 + }, + { + "budget": 0.63, + "threshold": -2.2176589965820312, + "operator": "ge", + "actual_fraction": 0.6177800100452034, + "metric": 0.9068666140489345, + "rescued": 1904, + "harmed": 415, + "missed_rescue": 343, + "avoided_harm": 430, + "selected_both_correct": 5827, + "selected_both_wrong": 464 + }, + { + "budget": 0.64, + "threshold": -2.2867987155914307, + "operator": "ge", + "actual_fraction": 0.6290449881610103, + "metric": 0.9070818684078352, + "rescued": 1918, + "harmed": 426, + "missed_rescue": 329, + "avoided_harm": 419, + "selected_both_correct": 5958, + "selected_both_wrong": 465 + }, + { + "budget": 0.65, + "threshold": -2.3520939350128174, + "operator": "ge", + "actual_fraction": 0.6400229604649494, + "metric": 0.9069383655019014, + "rescued": 1926, + "harmed": 436, + "missed_rescue": 321, + "avoided_harm": 409, + "selected_both_correct": 6088, + "selected_both_wrong": 470 + }, + { + "budget": 0.66, + "threshold": -2.4134349822998047, + "operator": "ge", + "actual_fraction": 0.650642175504054, + "metric": 0.9078711343904714, + "rescued": 1944, + "harmed": 441, + "missed_rescue": 303, + "avoided_harm": 404, + "selected_both_correct": 6209, + "selected_both_wrong": 474 + }, + { + "budget": 0.67, + "threshold": -2.4683873653411865, + "operator": "ge", + "actual_fraction": 0.6600416158427208, + "metric": 0.908229891655306, + "rescued": 1956, + "harmed": 448, + "missed_rescue": 291, + "avoided_harm": 397, + "selected_both_correct": 6320, + "selected_both_wrong": 475 + }, + { + "budget": 0.68, + "threshold": -2.5484306812286377, + "operator": "ge", + "actual_fraction": 0.672813374470833, + "metric": 0.9081581402023391, + "rescued": 1966, + "harmed": 459, + "missed_rescue": 281, + "avoided_harm": 386, + "selected_both_correct": 6475, + "selected_both_wrong": 477 + }, + { + "budget": 0.69, + "threshold": -2.608332633972168, + "operator": "ge", + "actual_fraction": 0.6819258089976322, + "metric": 0.9080863887493722, + "rescued": 1974, + "harmed": 468, + "missed_rescue": 273, + "avoided_harm": 377, + "selected_both_correct": 6583, + "selected_both_wrong": 479 + }, + { + "budget": 0.7, + "threshold": -2.670351505279541, + "operator": "ge", + "actual_fraction": 0.691325249336299, + "metric": 0.9080863887493722, + "rescued": 1985, + "harmed": 479, + "missed_rescue": 262, + "avoided_harm": 366, + "selected_both_correct": 6689, + "selected_both_wrong": 482 + }, + { + "budget": 0.71, + "threshold": -2.745088577270508, + "operator": "ge", + "actual_fraction": 0.7027337303580398, + "metric": 0.9084451460142068, + "rescued": 2003, + "harmed": 492, + "missed_rescue": 244, + "avoided_harm": 353, + "selected_both_correct": 6812, + "selected_both_wrong": 487 + }, + { + "budget": 0.72, + "threshold": -2.8100666999816895, + "operator": "ge", + "actual_fraction": 0.7129224366793427, + "metric": 0.9088756547320084, + "rescued": 2012, + "harmed": 495, + "missed_rescue": 235, + "avoided_harm": 350, + "selected_both_correct": 6942, + "selected_both_wrong": 487 + }, + { + "budget": 0.73, + "threshold": -2.867399215698242, + "operator": "ge", + "actual_fraction": 0.721963119753175, + "metric": 0.9084451460142068, + "rescued": 2016, + "harmed": 505, + "missed_rescue": 231, + "avoided_harm": 340, + "selected_both_correct": 7051, + "selected_both_wrong": 490 + }, + { + "budget": 0.74, + "threshold": -2.953387975692749, + "operator": "ge", + "actual_fraction": 0.7345913754753534, + "metric": 0.908301643108273, + "rescued": 2028, + "harmed": 519, + "missed_rescue": 219, + "avoided_harm": 326, + "selected_both_correct": 7199, + "selected_both_wrong": 492 + }, + { + "budget": 0.75, + "threshold": -3.0269675254821777, + "operator": "ge", + "actual_fraction": 0.7458563535911602, + "metric": 0.9089474061849753, + "rescued": 2041, + "harmed": 523, + "missed_rescue": 206, + "avoided_harm": 322, + "selected_both_correct": 7337, + "selected_both_wrong": 494 + }, + { + "budget": 0.76, + "threshold": -3.0986766815185547, + "operator": "ge", + "actual_fraction": 0.7564038171772979, + "metric": 0.9090191576379422, + "rescued": 2053, + "harmed": 534, + "missed_rescue": 194, + "avoided_harm": 311, + "selected_both_correct": 7460, + "selected_both_wrong": 495 + }, + { + "budget": 0.77, + "threshold": -3.183574676513672, + "operator": "ge", + "actual_fraction": 0.7665925234986009, + "metric": 0.9088039032790414, + "rescued": 2060, + "harmed": 544, + "missed_rescue": 187, + "avoided_harm": 301, + "selected_both_correct": 7581, + "selected_both_wrong": 499 + }, + { + "budget": 0.78, + "threshold": -3.2600371837615967, + "operator": "ge", + "actual_fraction": 0.7777139987084738, + "metric": 0.908229891655306, + "rescued": 2071, + "harmed": 563, + "missed_rescue": 176, + "avoided_harm": 282, + "selected_both_correct": 7704, + "selected_both_wrong": 501 + }, + { + "budget": 0.79, + "threshold": -3.3452396392822266, + "operator": "ge", + "actual_fraction": 0.7888354739183469, + "metric": 0.9079428858434383, + "rescued": 2080, + "harmed": 576, + "missed_rescue": 167, + "avoided_harm": 269, + "selected_both_correct": 7833, + "selected_both_wrong": 505 + }, + { + "budget": 0.8, + "threshold": -3.4057769775390625, + "operator": "ge", + "actual_fraction": 0.7967281337447083, + "metric": 0.9075841285786037, + "rescued": 2086, + "harmed": 587, + "missed_rescue": 161, + "avoided_harm": 258, + "selected_both_correct": 7924, + "selected_both_wrong": 507 + }, + { + "budget": 0.81, + "threshold": -3.474921464920044, + "operator": "ge", + "actual_fraction": 0.8036162732295329, + "metric": 0.9079428858434383, + "rescued": 2097, + "harmed": 593, + "missed_rescue": 150, + "avoided_harm": 252, + "selected_both_correct": 8002, + "selected_both_wrong": 508 + }, + { + "budget": 0.82, + "threshold": -3.5584769248962402, + "operator": "ge", + "actual_fraction": 0.8145224940805051, + "metric": 0.9076558800315706, + "rescued": 2104, + "harmed": 604, + "missed_rescue": 143, + "avoided_harm": 241, + "selected_both_correct": 8133, + "selected_both_wrong": 511 + }, + { + "budget": 0.83, + "threshold": -3.660430431365967, + "operator": "ge", + "actual_fraction": 0.8261462294611466, + "metric": 0.9078711343904714, + "rescued": 2114, + "harmed": 611, + "missed_rescue": 133, + "avoided_harm": 234, + "selected_both_correct": 8273, + "selected_both_wrong": 516 + }, + { + "budget": 0.84, + "threshold": -3.760495185852051, + "operator": "ge", + "actual_fraction": 0.836837195953218, + "metric": 0.9075123771256368, + "rescued": 2120, + "harmed": 622, + "missed_rescue": 127, + "avoided_harm": 223, + "selected_both_correct": 8405, + "selected_both_wrong": 516 + }, + { + "budget": 0.85, + "threshold": -3.8480324745178223, + "operator": "ge", + "actual_fraction": 0.8459496304800173, + "metric": 0.9074406256726699, + "rescued": 2131, + "harmed": 634, + "missed_rescue": 116, + "avoided_harm": 211, + "selected_both_correct": 8509, + "selected_both_wrong": 516 + }, + { + "budget": 0.86, + "threshold": -3.9325687885284424, + "operator": "ge", + "actual_fraction": 0.8561383368013202, + "metric": 0.9075123771256368, + "rescued": 2144, + "harmed": 646, + "missed_rescue": 103, + "avoided_harm": 199, + "selected_both_correct": 8626, + "selected_both_wrong": 516 + }, + { + "budget": 0.87, + "threshold": -4.00416374206543, + "operator": "ge", + "actual_fraction": 0.8643180024395494, + "metric": 0.9074406256726699, + "rescued": 2150, + "harmed": 653, + "missed_rescue": 97, + "avoided_harm": 192, + "selected_both_correct": 8726, + "selected_both_wrong": 517 + }, + { + "budget": 0.88, + "threshold": -4.082941055297852, + "operator": "ge", + "actual_fraction": 0.874004448590084, + "metric": 0.9073688742197029, + "rescued": 2159, + "harmed": 663, + "missed_rescue": 88, + "avoided_harm": 182, + "selected_both_correct": 8840, + "selected_both_wrong": 519 + }, + { + "budget": 0.89, + "threshold": -4.1705641746521, + "operator": "ge", + "actual_fraction": 0.8816818540575446, + "metric": 0.9070818684078352, + "rescued": 2162, + "harmed": 670, + "missed_rescue": 85, + "avoided_harm": 175, + "selected_both_correct": 8934, + "selected_both_wrong": 522 + }, + { + "budget": 0.9, + "threshold": -4.255984306335449, + "operator": "ge", + "actual_fraction": 0.8912247973021453, + "metric": 0.9070818684078352, + "rescued": 2172, + "harmed": 680, + "missed_rescue": 75, + "avoided_harm": 165, + "selected_both_correct": 9046, + "selected_both_wrong": 523 + }, + { + "budget": 0.91, + "threshold": -4.356119155883789, + "operator": "ge", + "actual_fraction": 0.901772260888283, + "metric": 0.9067948625959675, + "rescued": 2180, + "harmed": 692, + "missed_rescue": 67, + "avoided_harm": 153, + "selected_both_correct": 9173, + "selected_both_wrong": 523 + }, + { + "budget": 0.92, + "threshold": -4.483384132385254, + "operator": "ge", + "actual_fraction": 0.9164813087465021, + "metric": 0.9062208509722322, + "rescued": 2191, + "harmed": 711, + "missed_rescue": 56, + "avoided_harm": 134, + "selected_both_correct": 9346, + "selected_both_wrong": 525 + }, + { + "budget": 0.93, + "threshold": -4.5789947509765625, + "operator": "ge", + "actual_fraction": 0.926813517973739, + "metric": 0.9059338451603645, + "rescued": 2198, + "harmed": 722, + "missed_rescue": 49, + "avoided_harm": 123, + "selected_both_correct": 9469, + "selected_both_wrong": 528 + }, + { + "budget": 0.94, + "threshold": -4.691610336303711, + "operator": "ge", + "actual_fraction": 0.9372174786539428, + "metric": 0.905503336442563, + "rescued": 2206, + "harmed": 736, + "missed_rescue": 41, + "avoided_harm": 109, + "selected_both_correct": 9590, + "selected_both_wrong": 530 + }, + { + "budget": 0.95, + "threshold": -4.819090843200684, + "operator": "ge", + "actual_fraction": 0.947190930616345, + "metric": 0.9042835617421253, + "rescued": 2210, + "harmed": 757, + "missed_rescue": 37, + "avoided_harm": 88, + "selected_both_correct": 9703, + "selected_both_wrong": 531 + }, + { + "budget": 0.96, + "threshold": -4.976077079772949, + "operator": "ge", + "actual_fraction": 0.9598909377914903, + "metric": 0.9038530530243237, + "rescued": 2220, + "harmed": 773, + "missed_rescue": 27, + "avoided_harm": 72, + "selected_both_correct": 9852, + "selected_both_wrong": 533 + }, + { + "budget": 0.97, + "threshold": -5.13118839263916, + "operator": "ge", + "actual_fraction": 0.9698643897538926, + "metric": 0.9034942957594891, + "rescued": 2227, + "harmed": 785, + "missed_rescue": 20, + "avoided_harm": 60, + "selected_both_correct": 9970, + "selected_both_wrong": 535 + }, + { + "budget": 0.98, + "threshold": -5.317582130432129, + "operator": "ge", + "actual_fraction": 0.9799813446222286, + "metric": 0.9029920355887207, + "rescued": 2235, + "harmed": 800, + "missed_rescue": 12, + "avoided_harm": 45, + "selected_both_correct": 10088, + "selected_both_wrong": 535 + }, + { + "budget": 0.99, + "threshold": -5.640842437744141, + "operator": "ge", + "actual_fraction": 0.989811293678697, + "metric": 0.9021310181531176, + "rescued": 2238, + "harmed": 815, + "missed_rescue": 9, + "avoided_harm": 30, + "selected_both_correct": 10204, + "selected_both_wrong": 538 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9006242376408122, + "rescued": 2247, + "harmed": 845, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 10305, + "selected_both_wrong": 540 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 5165, + "rescuable": 1154, + "harmful": 408, + "both_wrong": 255, + "unit": "sample", + "total": 6982 + }, + "test": { + "both_correct": 10305, + "rescuable": 2247, + "harmful": 845, + "both_wrong": 540, + "unit": "sample", + "total": 13937 + } + }, + "small": { + "validation": 0.7981953594958464, + "test": 0.8000287005811868 + }, + "large": { + "validation": 0.9050415353766829, + "test": 0.9006242376408122 + }, + "learned_minus_confidence_test": 0.006385879314056098, + "learned_minus_small_test": 0.10877520269785457, + "learned_minus_large_test": 0.008179665638229139 + }, + { + "key": [ + "nlvr2", + "vilt", + "MiniCPM" + ], + "expert_display": "ViLT", + "run": "shared_12_vilt_MiniCPM", + "validation_rows": 6982, + "test_rows": 13937, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/vilt/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 96, + "validation_area_0_25": 0.7926095674591808, + "validation_point": { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9598968776854769, + "metric": 0.8997421942136924, + "rescued": 1385, + "harmed": 379, + "missed_rescue": 4, + "avoided_harm": 9, + "selected_both_correct": 4621, + "selected_both_wrong": 317 + }, + "test_point": { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9593169261677549, + "metric": 0.906436105331133, + "rescued": 2682, + "harmed": 700, + "missed_rescue": 24, + "avoided_harm": 11, + "selected_both_correct": 9413, + "selected_both_wrong": 575 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7556574047550845, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1389, + "avoided_harm": 388, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5127503275871277, + "operator": "ge", + "actual_fraction": 0.009882555141793183, + "metric": 0.7590948152391864, + "rescued": 31, + "harmed": 7, + "missed_rescue": 1358, + "avoided_harm": 381, + "selected_both_correct": 27, + "selected_both_wrong": 4 + }, + { + "budget": 0.02, + "threshold": -0.5296345353126526, + "operator": "ge", + "actual_fraction": 0.01990833572042395, + "metric": 0.7619593239759381, + "rescued": 55, + "harmed": 11, + "missed_rescue": 1334, + "avoided_harm": 377, + "selected_both_correct": 64, + "selected_both_wrong": 9 + }, + { + "budget": 0.03, + "threshold": -0.5434877276420593, + "operator": "ge", + "actual_fraction": 0.029934116299054712, + "metric": 0.7639644800916643, + "rescued": 80, + "harmed": 22, + "missed_rescue": 1309, + "avoided_harm": 366, + "selected_both_correct": 93, + "selected_both_wrong": 14 + }, + { + "budget": 0.04, + "threshold": -0.56125408411026, + "operator": "ge", + "actual_fraction": 0.039959896877685475, + "metric": 0.7672586651389287, + "rescued": 109, + "harmed": 28, + "missed_rescue": 1280, + "avoided_harm": 360, + "selected_both_correct": 124, + "selected_both_wrong": 18 + }, + { + "budget": 0.05, + "threshold": -0.5779195427894592, + "operator": "ge", + "actual_fraction": 0.04998567745631624, + "metric": 0.771268977370381, + "rescued": 137, + "harmed": 28, + "missed_rescue": 1252, + "avoided_harm": 360, + "selected_both_correct": 158, + "selected_both_wrong": 26 + }, + { + "budget": 0.06, + "threshold": -0.5911363363265991, + "operator": "ge", + "actual_fraction": 0.059868232598109423, + "metric": 0.7734173589229447, + "rescued": 160, + "harmed": 36, + "missed_rescue": 1229, + "avoided_harm": 352, + "selected_both_correct": 189, + "selected_both_wrong": 33 + }, + { + "budget": 0.07, + "threshold": -0.6087464094161987, + "operator": "ge", + "actual_fraction": 0.06989401317674018, + "metric": 0.7769979948438843, + "rescued": 192, + "harmed": 43, + "missed_rescue": 1197, + "avoided_harm": 345, + "selected_both_correct": 215, + "selected_both_wrong": 38 + }, + { + "budget": 0.08, + "threshold": -0.6274068355560303, + "operator": "ge", + "actual_fraction": 0.07991979375537095, + "metric": 0.7805786307648238, + "rescued": 223, + "harmed": 49, + "missed_rescue": 1166, + "avoided_harm": 339, + "selected_both_correct": 245, + "selected_both_wrong": 41 + }, + { + "budget": 0.09, + "threshold": -0.6465112566947937, + "operator": "ge", + "actual_fraction": 0.08994557433400172, + "metric": 0.783586364938413, + "rescued": 249, + "harmed": 54, + "missed_rescue": 1140, + "avoided_harm": 334, + "selected_both_correct": 279, + "selected_both_wrong": 46 + }, + { + "budget": 0.1, + "threshold": -0.6620025038719177, + "operator": "ge", + "actual_fraction": 0.09997135491263248, + "metric": 0.7877399026067029, + "rescued": 281, + "harmed": 57, + "missed_rescue": 1108, + "avoided_harm": 331, + "selected_both_correct": 307, + "selected_both_wrong": 53 + }, + { + "budget": 0.11, + "threshold": -0.6800539493560791, + "operator": "ge", + "actual_fraction": 0.10999713549126325, + "metric": 0.7901747350329419, + "rescued": 303, + "harmed": 62, + "missed_rescue": 1086, + "avoided_harm": 326, + "selected_both_correct": 345, + "selected_both_wrong": 58 + }, + { + "budget": 0.12, + "threshold": -0.6957284212112427, + "operator": "ge", + "actual_fraction": 0.11987969063305644, + "metric": 0.7926095674591808, + "rescued": 329, + "harmed": 71, + "missed_rescue": 1060, + "avoided_harm": 317, + "selected_both_correct": 372, + "selected_both_wrong": 65 + }, + { + "budget": 0.13, + "threshold": -0.7128273248672485, + "operator": "ge", + "actual_fraction": 0.1299054712116872, + "metric": 0.7954740761959324, + "rescued": 355, + "harmed": 77, + "missed_rescue": 1034, + "avoided_harm": 311, + "selected_both_correct": 405, + "selected_both_wrong": 70 + }, + { + "budget": 0.14, + "threshold": -0.7332167625427246, + "operator": "ge", + "actual_fraction": 0.13993125179031796, + "metric": 0.7977656831853337, + "rescued": 379, + "harmed": 85, + "missed_rescue": 1010, + "avoided_harm": 303, + "selected_both_correct": 439, + "selected_both_wrong": 74 + }, + { + "budget": 0.15, + "threshold": -0.7503995895385742, + "operator": "ge", + "actual_fraction": 0.14995703236894872, + "metric": 0.8010598682325981, + "rescued": 408, + "harmed": 91, + "missed_rescue": 981, + "avoided_harm": 297, + "selected_both_correct": 468, + "selected_both_wrong": 80 + }, + { + "budget": 0.16, + "threshold": -0.7655540704727173, + "operator": "ge", + "actual_fraction": 0.1599828129475795, + "metric": 0.8026353480378116, + "rescued": 428, + "harmed": 100, + "missed_rescue": 961, + "avoided_harm": 288, + "selected_both_correct": 507, + "selected_both_wrong": 82 + }, + { + "budget": 0.17, + "threshold": -0.7812193036079407, + "operator": "ge", + "actual_fraction": 0.16986536808937266, + "metric": 0.8059295330850759, + "rescued": 453, + "harmed": 102, + "missed_rescue": 936, + "avoided_harm": 286, + "selected_both_correct": 543, + "selected_both_wrong": 88 + }, + { + "budget": 0.18, + "threshold": -0.7954917550086975, + "operator": "ge", + "actual_fraction": 0.17989114866800343, + "metric": 0.806932111142939, + "rescued": 470, + "harmed": 112, + "missed_rescue": 919, + "avoided_harm": 276, + "selected_both_correct": 584, + "selected_both_wrong": 90 + }, + { + "budget": 0.19, + "threshold": -0.8097342252731323, + "operator": "ge", + "actual_fraction": 0.1899169292466342, + "metric": 0.8093669435691779, + "rescued": 494, + "harmed": 119, + "missed_rescue": 895, + "avoided_harm": 269, + "selected_both_correct": 619, + "selected_both_wrong": 94 + }, + { + "budget": 0.2, + "threshold": -0.8232117891311646, + "operator": "ge", + "actual_fraction": 0.19994270982526496, + "metric": 0.8122314523059295, + "rescued": 517, + "harmed": 122, + "missed_rescue": 872, + "avoided_harm": 266, + "selected_both_correct": 658, + "selected_both_wrong": 99 + }, + { + "budget": 0.21, + "threshold": -0.8359466791152954, + "operator": "ge", + "actual_fraction": 0.20996849040389573, + "metric": 0.8146662847321684, + "rescued": 539, + "harmed": 127, + "missed_rescue": 850, + "avoided_harm": 261, + "selected_both_correct": 697, + "selected_both_wrong": 103 + }, + { + "budget": 0.22, + "threshold": -0.8473842144012451, + "operator": "ge", + "actual_fraction": 0.2199942709825265, + "metric": 0.8168146662847322, + "rescued": 560, + "harmed": 133, + "missed_rescue": 829, + "avoided_harm": 255, + "selected_both_correct": 734, + "selected_both_wrong": 109 + }, + { + "budget": 0.23, + "threshold": -0.8595863580703735, + "operator": "ge", + "actual_fraction": 0.22987682612431967, + "metric": 0.8203953022056717, + "rescued": 588, + "harmed": 136, + "missed_rescue": 801, + "avoided_harm": 252, + "selected_both_correct": 767, + "selected_both_wrong": 114 + }, + { + "budget": 0.24, + "threshold": -0.8704345226287842, + "operator": "ge", + "actual_fraction": 0.23990260670295044, + "metric": 0.8225436837582355, + "rescued": 610, + "harmed": 143, + "missed_rescue": 779, + "avoided_harm": 245, + "selected_both_correct": 805, + "selected_both_wrong": 117 + }, + { + "budget": 0.25, + "threshold": -0.8811548352241516, + "operator": "ge", + "actual_fraction": 0.2499283872815812, + "metric": 0.8258378688054998, + "rescued": 637, + "harmed": 147, + "missed_rescue": 752, + "avoided_harm": 241, + "selected_both_correct": 836, + "selected_both_wrong": 125 + }, + { + "budget": 0.26, + "threshold": -0.8929013609886169, + "operator": "ge", + "actual_fraction": 0.25995416786021197, + "metric": 0.8279862503580636, + "rescued": 656, + "harmed": 151, + "missed_rescue": 733, + "avoided_harm": 237, + "selected_both_correct": 880, + "selected_both_wrong": 128 + }, + { + "budget": 0.27, + "threshold": -0.903549313545227, + "operator": "ge", + "actual_fraction": 0.26997994843884277, + "metric": 0.829561730163277, + "rescued": 675, + "harmed": 159, + "missed_rescue": 714, + "avoided_harm": 229, + "selected_both_correct": 915, + "selected_both_wrong": 136 + }, + { + "budget": 0.28, + "threshold": -0.9115740656852722, + "operator": "ge", + "actual_fraction": 0.2798625035806359, + "metric": 0.8321397880263535, + "rescued": 697, + "harmed": 163, + "missed_rescue": 692, + "avoided_harm": 225, + "selected_both_correct": 953, + "selected_both_wrong": 141 + }, + { + "budget": 0.29, + "threshold": -0.9182595014572144, + "operator": "ge", + "actual_fraction": 0.2898882841592667, + "metric": 0.83471784588943, + "rescued": 718, + "harmed": 166, + "missed_rescue": 671, + "avoided_harm": 222, + "selected_both_correct": 997, + "selected_both_wrong": 143 + }, + { + "budget": 0.3, + "threshold": -0.9258768558502197, + "operator": "ge", + "actual_fraction": 0.29991406473789745, + "metric": 0.8361501002578058, + "rescued": 735, + "harmed": 173, + "missed_rescue": 654, + "avoided_harm": 215, + "selected_both_correct": 1041, + "selected_both_wrong": 145 + }, + { + "budget": 0.31, + "threshold": -0.9331877827644348, + "operator": "ge", + "actual_fraction": 0.30993984531652824, + "metric": 0.8381552563735319, + "rescued": 754, + "harmed": 178, + "missed_rescue": 635, + "avoided_harm": 210, + "selected_both_correct": 1080, + "selected_both_wrong": 152 + }, + { + "budget": 0.32, + "threshold": -0.9399217367172241, + "operator": "ge", + "actual_fraction": 0.319965625895159, + "metric": 0.8405900887997708, + "rescued": 774, + "harmed": 181, + "missed_rescue": 615, + "avoided_harm": 207, + "selected_both_correct": 1123, + "selected_both_wrong": 156 + }, + { + "budget": 0.33, + "threshold": -0.9453923106193542, + "operator": "ge", + "actual_fraction": 0.3299914064737898, + "metric": 0.8423087940418218, + "rescued": 792, + "harmed": 187, + "missed_rescue": 597, + "avoided_harm": 201, + "selected_both_correct": 1162, + "selected_both_wrong": 163 + }, + { + "budget": 0.34, + "threshold": -0.9515207409858704, + "operator": "ge", + "actual_fraction": 0.3398739616155829, + "metric": 0.8438842738470352, + "rescued": 807, + "harmed": 191, + "missed_rescue": 582, + "avoided_harm": 197, + "selected_both_correct": 1206, + "selected_both_wrong": 169 + }, + { + "budget": 0.35, + "threshold": -0.9567410349845886, + "operator": "ge", + "actual_fraction": 0.3498997421942137, + "metric": 0.8466055571469493, + "rescued": 829, + "harmed": 194, + "missed_rescue": 560, + "avoided_harm": 194, + "selected_both_correct": 1245, + "selected_both_wrong": 175 + }, + { + "budget": 0.36, + "threshold": -0.9606627225875854, + "operator": "ge", + "actual_fraction": 0.35992552277284445, + "metric": 0.8498997421942137, + "rescued": 854, + "harmed": 196, + "missed_rescue": 535, + "avoided_harm": 192, + "selected_both_correct": 1287, + "selected_both_wrong": 176 + }, + { + "budget": 0.37, + "threshold": -0.9650874733924866, + "operator": "ge", + "actual_fraction": 0.36995130335147525, + "metric": 0.8519048983099399, + "rescued": 872, + "harmed": 200, + "missed_rescue": 517, + "avoided_harm": 188, + "selected_both_correct": 1329, + "selected_both_wrong": 182 + }, + { + "budget": 0.38, + "threshold": -0.9686882495880127, + "operator": "ge", + "actual_fraction": 0.379977083930106, + "metric": 0.8547694070466915, + "rescued": 893, + "harmed": 201, + "missed_rescue": 496, + "avoided_harm": 187, + "selected_both_correct": 1372, + "selected_both_wrong": 187 + }, + { + "budget": 0.39, + "threshold": -0.971785306930542, + "operator": "ge", + "actual_fraction": 0.3898596390718992, + "metric": 0.8557719851045545, + "rescued": 903, + "harmed": 204, + "missed_rescue": 486, + "avoided_harm": 184, + "selected_both_correct": 1423, + "selected_both_wrong": 192 + }, + { + "budget": 0.4, + "threshold": -0.9748144745826721, + "operator": "ge", + "actual_fraction": 0.3998854196505299, + "metric": 0.8584932684044686, + "rescued": 923, + "harmed": 205, + "missed_rescue": 466, + "avoided_harm": 183, + "selected_both_correct": 1468, + "selected_both_wrong": 196 + }, + { + "budget": 0.41, + "threshold": -0.977255642414093, + "operator": "ge", + "actual_fraction": 0.4099112002291607, + "metric": 0.8596390718991693, + "rescued": 935, + "harmed": 209, + "missed_rescue": 454, + "avoided_harm": 179, + "selected_both_correct": 1520, + "selected_both_wrong": 198 + }, + { + "budget": 0.42, + "threshold": -0.9794421792030334, + "operator": "ge", + "actual_fraction": 0.41993698080779146, + "metric": 0.861787453451733, + "rescued": 954, + "harmed": 213, + "missed_rescue": 435, + "avoided_harm": 175, + "selected_both_correct": 1560, + "selected_both_wrong": 205 + }, + { + "budget": 0.43, + "threshold": -0.981472373008728, + "operator": "ge", + "actual_fraction": 0.42996276138642225, + "metric": 0.8632197078201088, + "rescued": 969, + "harmed": 218, + "missed_rescue": 420, + "avoided_harm": 170, + "selected_both_correct": 1606, + "selected_both_wrong": 209 + }, + { + "budget": 0.44, + "threshold": -0.9835578799247742, + "operator": "ge", + "actual_fraction": 0.439988541965053, + "metric": 0.8649384130621598, + "rescued": 986, + "harmed": 223, + "missed_rescue": 403, + "avoided_harm": 165, + "selected_both_correct": 1653, + "selected_both_wrong": 210 + }, + { + "budget": 0.45, + "threshold": -0.9851564168930054, + "operator": "ge", + "actual_fraction": 0.4498710971068462, + "metric": 0.8663706674305357, + "rescued": 1001, + "harmed": 228, + "missed_rescue": 388, + "avoided_harm": 160, + "selected_both_correct": 1699, + "selected_both_wrong": 213 + }, + { + "budget": 0.46, + "threshold": -0.986689031124115, + "operator": "ge", + "actual_fraction": 0.45989687768547693, + "metric": 0.8673732454883988, + "rescued": 1016, + "harmed": 236, + "missed_rescue": 373, + "avoided_harm": 152, + "selected_both_correct": 1742, + "selected_both_wrong": 217 + }, + { + "budget": 0.47, + "threshold": -0.9879001379013062, + "operator": "ge", + "actual_fraction": 0.46992265826410773, + "metric": 0.8680893726725867, + "rescued": 1024, + "harmed": 239, + "missed_rescue": 365, + "avoided_harm": 149, + "selected_both_correct": 1797, + "selected_both_wrong": 221 + }, + { + "budget": 0.48, + "threshold": -0.9891784191131592, + "operator": "ge", + "actual_fraction": 0.47994843884273847, + "metric": 0.8700945287883128, + "rescued": 1043, + "harmed": 244, + "missed_rescue": 346, + "avoided_harm": 144, + "selected_both_correct": 1839, + "selected_both_wrong": 225 + }, + { + "budget": 0.49, + "threshold": -0.990114688873291, + "operator": "ge", + "actual_fraction": 0.48997421942136926, + "metric": 0.8715267831566886, + "rescued": 1057, + "harmed": 248, + "missed_rescue": 332, + "avoided_harm": 140, + "selected_both_correct": 1889, + "selected_both_wrong": 227 + }, + { + "budget": 0.5, + "threshold": -0.9910277724266052, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.8722429103408765, + "rescued": 1066, + "harmed": 252, + "missed_rescue": 323, + "avoided_harm": 136, + "selected_both_correct": 1942, + "selected_both_wrong": 231 + }, + { + "budget": 0.51, + "threshold": -0.9918718934059143, + "operator": "ge", + "actual_fraction": 0.5098825551417931, + "metric": 0.8725293612145517, + "rescued": 1074, + "harmed": 258, + "missed_rescue": 315, + "avoided_harm": 130, + "selected_both_correct": 1996, + "selected_both_wrong": 232 + }, + { + "budget": 0.52, + "threshold": -0.9926809668540955, + "operator": "ge", + "actual_fraction": 0.5199083357204239, + "metric": 0.8729590375250644, + "rescued": 1083, + "harmed": 264, + "missed_rescue": 306, + "avoided_harm": 124, + "selected_both_correct": 2050, + "selected_both_wrong": 233 + }, + { + "budget": 0.53, + "threshold": -0.9932646751403809, + "operator": "ge", + "actual_fraction": 0.5299341162990547, + "metric": 0.87381839014609, + "rescued": 1090, + "harmed": 265, + "missed_rescue": 299, + "avoided_harm": 123, + "selected_both_correct": 2108, + "selected_both_wrong": 237 + }, + { + "budget": 0.54, + "threshold": -0.9938417673110962, + "operator": "ge", + "actual_fraction": 0.5399598968776855, + "metric": 0.8743912918934402, + "rescued": 1097, + "harmed": 268, + "missed_rescue": 292, + "avoided_harm": 120, + "selected_both_correct": 2166, + "selected_both_wrong": 239 + }, + { + "budget": 0.55, + "threshold": -0.9944412112236023, + "operator": "ge", + "actual_fraction": 0.5499856774563162, + "metric": 0.8751074190776282, + "rescued": 1107, + "harmed": 273, + "missed_rescue": 282, + "avoided_harm": 115, + "selected_both_correct": 2220, + "selected_both_wrong": 240 + }, + { + "budget": 0.56, + "threshold": -0.9949513673782349, + "operator": "ge", + "actual_fraction": 0.5598682325981095, + "metric": 0.8761099971354913, + "rescued": 1118, + "harmed": 277, + "missed_rescue": 271, + "avoided_harm": 111, + "selected_both_correct": 2272, + "selected_both_wrong": 242 + }, + { + "budget": 0.57, + "threshold": -0.9954326152801514, + "operator": "ge", + "actual_fraction": 0.5698940131767402, + "metric": 0.8776854769407046, + "rescued": 1131, + "harmed": 279, + "missed_rescue": 258, + "avoided_harm": 109, + "selected_both_correct": 2323, + "selected_both_wrong": 246 + }, + { + "budget": 0.58, + "threshold": -0.9958608746528625, + "operator": "ge", + "actual_fraction": 0.579919793755371, + "metric": 0.878258378688055, + "rescued": 1142, + "harmed": 286, + "missed_rescue": 247, + "avoided_harm": 102, + "selected_both_correct": 2371, + "selected_both_wrong": 250 + }, + { + "budget": 0.59, + "threshold": -0.9962128400802612, + "operator": "ge", + "actual_fraction": 0.5899455743340017, + "metric": 0.8792609567459181, + "rescued": 1154, + "harmed": 291, + "missed_rescue": 235, + "avoided_harm": 97, + "selected_both_correct": 2421, + "selected_both_wrong": 253 + }, + { + "budget": 0.6, + "threshold": -0.9965764880180359, + "operator": "ge", + "actual_fraction": 0.5999713549126325, + "metric": 0.8796906330564308, + "rescued": 1161, + "harmed": 295, + "missed_rescue": 228, + "avoided_harm": 93, + "selected_both_correct": 2479, + "selected_both_wrong": 254 + }, + { + "budget": 0.61, + "threshold": -0.9968799352645874, + "operator": "ge", + "actual_fraction": 0.6099971354912632, + "metric": 0.8808364365511315, + "rescued": 1174, + "harmed": 300, + "missed_rescue": 215, + "avoided_harm": 88, + "selected_both_correct": 2530, + "selected_both_wrong": 255 + }, + { + "budget": 0.62, + "threshold": -0.9971383810043335, + "operator": "ge", + "actual_fraction": 0.6198796906330565, + "metric": 0.8811228874248066, + "rescued": 1178, + "harmed": 302, + "missed_rescue": 211, + "avoided_harm": 86, + "selected_both_correct": 2591, + "selected_both_wrong": 257 + }, + { + "budget": 0.63, + "threshold": -0.997377872467041, + "operator": "ge", + "actual_fraction": 0.6299054712116872, + "metric": 0.8822686909195073, + "rescued": 1188, + "harmed": 304, + "missed_rescue": 201, + "avoided_harm": 84, + "selected_both_correct": 2645, + "selected_both_wrong": 261 + }, + { + "budget": 0.64, + "threshold": -0.9976021647453308, + "operator": "ge", + "actual_fraction": 0.639931251790318, + "metric": 0.8832712689773704, + "rescued": 1200, + "harmed": 309, + "missed_rescue": 189, + "avoided_harm": 79, + "selected_both_correct": 2695, + "selected_both_wrong": 264 + }, + { + "budget": 0.65, + "threshold": -0.9978214502334595, + "operator": "ge", + "actual_fraction": 0.6499570323689488, + "metric": 0.8834144944142079, + "rescued": 1205, + "harmed": 313, + "missed_rescue": 184, + "avoided_harm": 75, + "selected_both_correct": 2753, + "selected_both_wrong": 267 + }, + { + "budget": 0.66, + "threshold": -0.9980514049530029, + "operator": "ge", + "actual_fraction": 0.6599828129475795, + "metric": 0.8847035233457462, + "rescued": 1215, + "harmed": 314, + "missed_rescue": 174, + "avoided_harm": 74, + "selected_both_correct": 2810, + "selected_both_wrong": 269 + }, + { + "budget": 0.67, + "threshold": -0.9982016086578369, + "operator": "ge", + "actual_fraction": 0.6698653680893727, + "metric": 0.8855628759667717, + "rescued": 1223, + "harmed": 316, + "missed_rescue": 166, + "avoided_harm": 72, + "selected_both_correct": 2867, + "selected_both_wrong": 271 + }, + { + "budget": 0.68, + "threshold": -0.9983693957328796, + "operator": "ge", + "actual_fraction": 0.6798911486680035, + "metric": 0.8862790031509596, + "rescued": 1231, + "harmed": 319, + "missed_rescue": 158, + "avoided_harm": 69, + "selected_both_correct": 2925, + "selected_both_wrong": 272 + }, + { + "budget": 0.69, + "threshold": -0.9984915256500244, + "operator": "ge", + "actual_fraction": 0.6899169292466342, + "metric": 0.887854482956173, + "rescued": 1243, + "harmed": 320, + "missed_rescue": 146, + "avoided_harm": 68, + "selected_both_correct": 2980, + "selected_both_wrong": 274 + }, + { + "budget": 0.7, + "threshold": -0.9986109733581543, + "operator": "gt", + "actual_fraction": 0.6997994843884274, + "metric": 0.8881409338298482, + "rescued": 1248, + "harmed": 323, + "missed_rescue": 141, + "avoided_harm": 65, + "selected_both_correct": 3039, + "selected_both_wrong": 276 + }, + { + "budget": 0.71, + "threshold": -0.9986966252326965, + "operator": "ge", + "actual_fraction": 0.7099684904038958, + "metric": 0.888857061014036, + "rescued": 1253, + "harmed": 323, + "missed_rescue": 136, + "avoided_harm": 65, + "selected_both_correct": 3105, + "selected_both_wrong": 276 + }, + { + "budget": 0.72, + "threshold": -0.9987963438034058, + "operator": "ge", + "actual_fraction": 0.7199942709825266, + "metric": 0.8890002864508737, + "rescued": 1258, + "harmed": 327, + "missed_rescue": 131, + "avoided_harm": 61, + "selected_both_correct": 3164, + "selected_both_wrong": 278 + }, + { + "budget": 0.73, + "threshold": -0.998875081539154, + "operator": "ge", + "actual_fraction": 0.7298768261243197, + "metric": 0.8897164136350616, + "rescued": 1264, + "harmed": 328, + "missed_rescue": 125, + "avoided_harm": 60, + "selected_both_correct": 3223, + "selected_both_wrong": 281 + }, + { + "budget": 0.74, + "threshold": -0.9989551305770874, + "operator": "ge", + "actual_fraction": 0.7399026067029505, + "metric": 0.8901460899455743, + "rescued": 1272, + "harmed": 333, + "missed_rescue": 117, + "avoided_harm": 55, + "selected_both_correct": 3277, + "selected_both_wrong": 284 + }, + { + "budget": 0.75, + "threshold": -0.9990480542182922, + "operator": "ge", + "actual_fraction": 0.7499283872815812, + "metric": 0.8911486680034374, + "rescued": 1280, + "harmed": 334, + "missed_rescue": 109, + "avoided_harm": 54, + "selected_both_correct": 3335, + "selected_both_wrong": 287 + }, + { + "budget": 0.76, + "threshold": -0.999126136302948, + "operator": "ge", + "actual_fraction": 0.759954167860212, + "metric": 0.8915783443139501, + "rescued": 1286, + "harmed": 337, + "missed_rescue": 103, + "avoided_harm": 51, + "selected_both_correct": 3393, + "selected_both_wrong": 290 + }, + { + "budget": 0.77, + "threshold": -0.999184787273407, + "operator": "gt", + "actual_fraction": 0.7698367230020051, + "metric": 0.8920080206244629, + "rescued": 1291, + "harmed": 339, + "missed_rescue": 98, + "avoided_harm": 49, + "selected_both_correct": 3454, + "selected_both_wrong": 291 + }, + { + "budget": 0.78, + "threshold": -0.9992268085479736, + "operator": "ge", + "actual_fraction": 0.7798625035806359, + "metric": 0.893010598682326, + "rescued": 1301, + "harmed": 342, + "missed_rescue": 88, + "avoided_harm": 46, + "selected_both_correct": 3510, + "selected_both_wrong": 292 + }, + { + "budget": 0.79, + "threshold": -0.9992831349372864, + "operator": "ge", + "actual_fraction": 0.7898882841592667, + "metric": 0.8941564021770266, + "rescued": 1312, + "harmed": 345, + "missed_rescue": 77, + "avoided_harm": 43, + "selected_both_correct": 3564, + "selected_both_wrong": 294 + }, + { + "budget": 0.8, + "threshold": -0.9993318915367126, + "operator": "ge", + "actual_fraction": 0.7999140647378975, + "metric": 0.8942996276138642, + "rescued": 1314, + "harmed": 346, + "missed_rescue": 75, + "avoided_harm": 42, + "selected_both_correct": 3631, + "selected_both_wrong": 294 + }, + { + "budget": 0.81, + "threshold": -0.9993773102760315, + "operator": "ge", + "actual_fraction": 0.8099398453165282, + "metric": 0.8944428530507018, + "rescued": 1318, + "harmed": 349, + "missed_rescue": 71, + "avoided_harm": 39, + "selected_both_correct": 3693, + "selected_both_wrong": 295 + }, + { + "budget": 0.82, + "threshold": -0.9994175434112549, + "operator": "ge", + "actual_fraction": 0.819965625895159, + "metric": 0.8945860784875393, + "rescued": 1323, + "harmed": 353, + "missed_rescue": 66, + "avoided_harm": 35, + "selected_both_correct": 3752, + "selected_both_wrong": 297 + }, + { + "budget": 0.83, + "threshold": -0.999452531337738, + "operator": "ge", + "actual_fraction": 0.8299914064737898, + "metric": 0.894729303924377, + "rescued": 1327, + "harmed": 356, + "missed_rescue": 62, + "avoided_harm": 32, + "selected_both_correct": 3815, + "selected_both_wrong": 297 + }, + { + "budget": 0.84, + "threshold": -0.9994945526123047, + "operator": "ge", + "actual_fraction": 0.8398739616155829, + "metric": 0.8951589802348897, + "rescued": 1332, + "harmed": 358, + "missed_rescue": 57, + "avoided_harm": 30, + "selected_both_correct": 3876, + "selected_both_wrong": 298 + }, + { + "budget": 0.85, + "threshold": -0.9995274543762207, + "operator": "gt", + "actual_fraction": 0.8497565167573761, + "metric": 0.8954454311085649, + "rescued": 1335, + "harmed": 359, + "missed_rescue": 54, + "avoided_harm": 29, + "selected_both_correct": 3938, + "selected_both_wrong": 301 + }, + { + "budget": 0.86, + "threshold": -0.9995617270469666, + "operator": "ge", + "actual_fraction": 0.8599255227728445, + "metric": 0.8958751074190776, + "rescued": 1340, + "harmed": 361, + "missed_rescue": 49, + "avoided_harm": 27, + "selected_both_correct": 3999, + "selected_both_wrong": 304 + }, + { + "budget": 0.87, + "threshold": -0.9995949864387512, + "operator": "ge", + "actual_fraction": 0.8699513033514752, + "metric": 0.896448009166428, + "rescued": 1347, + "harmed": 364, + "missed_rescue": 42, + "avoided_harm": 24, + "selected_both_correct": 4059, + "selected_both_wrong": 304 + }, + { + "budget": 0.88, + "threshold": -0.9996248483657837, + "operator": "ge", + "actual_fraction": 0.879977083930106, + "metric": 0.8968776854769407, + "rescued": 1351, + "harmed": 365, + "missed_rescue": 38, + "avoided_harm": 23, + "selected_both_correct": 4123, + "selected_both_wrong": 305 + }, + { + "budget": 0.89, + "threshold": -0.9996511936187744, + "operator": "ge", + "actual_fraction": 0.8898596390718991, + "metric": 0.897450587224291, + "rescued": 1358, + "harmed": 368, + "missed_rescue": 31, + "avoided_harm": 20, + "selected_both_correct": 4180, + "selected_both_wrong": 307 + }, + { + "budget": 0.9, + "threshold": -0.9996830224990845, + "operator": "ge", + "actual_fraction": 0.8998854196505299, + "metric": 0.8977370380979662, + "rescued": 1361, + "harmed": 369, + "missed_rescue": 28, + "avoided_harm": 19, + "selected_both_correct": 4244, + "selected_both_wrong": 309 + }, + { + "budget": 0.91, + "threshold": -0.9997097849845886, + "operator": "gt", + "actual_fraction": 0.9097679747923231, + "metric": 0.8984531652821541, + "rescued": 1369, + "harmed": 372, + "missed_rescue": 20, + "avoided_harm": 16, + "selected_both_correct": 4302, + "selected_both_wrong": 309 + }, + { + "budget": 0.92, + "threshold": -0.9997391104698181, + "operator": "ge", + "actual_fraction": 0.9199369808077915, + "metric": 0.8985963907189917, + "rescued": 1372, + "harmed": 374, + "missed_rescue": 17, + "avoided_harm": 14, + "selected_both_correct": 4367, + "selected_both_wrong": 310 + }, + { + "budget": 0.93, + "threshold": -0.9997641444206238, + "operator": "ge", + "actual_fraction": 0.9299627613864222, + "metric": 0.8987396161558293, + "rescued": 1374, + "harmed": 375, + "missed_rescue": 15, + "avoided_harm": 13, + "selected_both_correct": 4429, + "selected_both_wrong": 315 + }, + { + "budget": 0.94, + "threshold": -0.9997876286506653, + "operator": "ge", + "actual_fraction": 0.939988541965053, + "metric": 0.8990260670295045, + "rescued": 1377, + "harmed": 376, + "missed_rescue": 12, + "avoided_harm": 12, + "selected_both_correct": 4494, + "selected_both_wrong": 316 + }, + { + "budget": 0.95, + "threshold": -0.9998131394386292, + "operator": "ge", + "actual_fraction": 0.9498710971068461, + "metric": 0.899169292466342, + "rescued": 1380, + "harmed": 378, + "missed_rescue": 9, + "avoided_harm": 10, + "selected_both_correct": 4557, + "selected_both_wrong": 317 + }, + { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9598968776854769, + "metric": 0.8997421942136924, + "rescued": 1385, + "harmed": 379, + "missed_rescue": 4, + "avoided_harm": 9, + "selected_both_correct": 4621, + "selected_both_wrong": 317 + }, + { + "budget": 0.97, + "threshold": -0.999868631362915, + "operator": "ge", + "actual_fraction": 0.9699226582641077, + "metric": 0.8993125179031796, + "rescued": 1385, + "harmed": 382, + "missed_rescue": 4, + "avoided_harm": 6, + "selected_both_correct": 4688, + "selected_both_wrong": 317 + }, + { + "budget": 0.98, + "threshold": -0.9998925924301147, + "operator": "ge", + "actual_fraction": 0.9799484388427385, + "metric": 0.899169292466342, + "rescued": 1385, + "harmed": 383, + "missed_rescue": 4, + "avoided_harm": 5, + "selected_both_correct": 4757, + "selected_both_wrong": 317 + }, + { + "budget": 0.99, + "threshold": -0.99992835521698, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.8987396161558293, + "rescued": 1385, + "harmed": 386, + "missed_rescue": 4, + "avoided_harm": 2, + "selected_both_correct": 4824, + "selected_both_wrong": 317 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8990260670295045, + "rescued": 1389, + "harmed": 388, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 4888, + "selected_both_wrong": 317 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7642247255506924, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2706, + "avoided_harm": 711, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5127503275871277, + "operator": "ge", + "actual_fraction": 0.007749156920427639, + "metric": 0.7658032575159647, + "rescued": 33, + "harmed": 11, + "missed_rescue": 2673, + "avoided_harm": 700, + "selected_both_correct": 57, + "selected_both_wrong": 7 + }, + { + "budget": 0.02, + "threshold": -0.5296345353126526, + "operator": "ge", + "actual_fraction": 0.017650857429862953, + "metric": 0.7694625816172778, + "rescued": 96, + "harmed": 23, + "missed_rescue": 2610, + "avoided_harm": 688, + "selected_both_correct": 117, + "selected_both_wrong": 10 + }, + { + "budget": 0.03, + "threshold": -0.5434877276420593, + "operator": "ge", + "actual_fraction": 0.025113008538422903, + "metric": 0.7724043911889216, + "rescued": 142, + "harmed": 28, + "missed_rescue": 2564, + "avoided_harm": 683, + "selected_both_correct": 160, + "selected_both_wrong": 20 + }, + { + "budget": 0.04, + "threshold": -0.56125408411026, + "operator": "ge", + "actual_fraction": 0.0349429575948913, + "metric": 0.7759919638372677, + "rescued": 199, + "harmed": 35, + "missed_rescue": 2507, + "avoided_harm": 676, + "selected_both_correct": 224, + "selected_both_wrong": 29 + }, + { + "budget": 0.05, + "threshold": -0.5779195427894592, + "operator": "ge", + "actual_fraction": 0.043840137762789694, + "metric": 0.7782880103322092, + "rescued": 238, + "harmed": 42, + "missed_rescue": 2468, + "avoided_harm": 669, + "selected_both_correct": 291, + "selected_both_wrong": 40 + }, + { + "budget": 0.06, + "threshold": -0.5911363363265991, + "operator": "ge", + "actual_fraction": 0.05187630049508503, + "metric": 0.7806558082801177, + "rescued": 281, + "harmed": 52, + "missed_rescue": 2425, + "avoided_harm": 659, + "selected_both_correct": 346, + "selected_both_wrong": 44 + }, + { + "budget": 0.07, + "threshold": -0.6087464094161987, + "operator": "ge", + "actual_fraction": 0.06177800100452034, + "metric": 0.7828801033220922, + "rescued": 326, + "harmed": 66, + "missed_rescue": 2380, + "avoided_harm": 645, + "selected_both_correct": 409, + "selected_both_wrong": 60 + }, + { + "budget": 0.08, + "threshold": -0.6274068355560303, + "operator": "ge", + "actual_fraction": 0.07132094424912104, + "metric": 0.7859654157996699, + "rescued": 376, + "harmed": 73, + "missed_rescue": 2330, + "avoided_harm": 638, + "selected_both_correct": 477, + "selected_both_wrong": 68 + }, + { + "budget": 0.09, + "threshold": -0.6465112566947937, + "operator": "ge", + "actual_fraction": 0.08251417091196096, + "metric": 0.788691971012413, + "rescued": 423, + "harmed": 82, + "missed_rescue": 2283, + "avoided_harm": 629, + "selected_both_correct": 560, + "selected_both_wrong": 85 + }, + { + "budget": 0.1, + "threshold": -0.6620025038719177, + "operator": "ge", + "actual_fraction": 0.09090909090909091, + "metric": 0.7908445146014207, + "rescued": 462, + "harmed": 91, + "missed_rescue": 2244, + "avoided_harm": 620, + "selected_both_correct": 619, + "selected_both_wrong": 95 + }, + { + "budget": 0.11, + "threshold": -0.6800539493560791, + "operator": "ge", + "actual_fraction": 0.10030853124775777, + "metric": 0.7934275669082299, + "rescued": 508, + "harmed": 101, + "missed_rescue": 2198, + "avoided_harm": 610, + "selected_both_correct": 682, + "selected_both_wrong": 107 + }, + { + "budget": 0.12, + "threshold": -0.6957284212112427, + "operator": "ge", + "actual_fraction": 0.11078424338092846, + "metric": 0.7968716366506422, + "rescued": 564, + "harmed": 109, + "missed_rescue": 2142, + "avoided_harm": 602, + "selected_both_correct": 747, + "selected_both_wrong": 124 + }, + { + "budget": 0.13, + "threshold": -0.7128273248672485, + "operator": "ge", + "actual_fraction": 0.12075769534333071, + "metric": 0.7998851976752529, + "rescued": 613, + "harmed": 116, + "missed_rescue": 2093, + "avoided_harm": 595, + "selected_both_correct": 821, + "selected_both_wrong": 133 + }, + { + "budget": 0.14, + "threshold": -0.7332167625427246, + "operator": "ge", + "actual_fraction": 0.13431871995407907, + "metric": 0.8045490421181029, + "rescued": 690, + "harmed": 128, + "missed_rescue": 2016, + "avoided_harm": 583, + "selected_both_correct": 902, + "selected_both_wrong": 152 + }, + { + "budget": 0.15, + "threshold": -0.7503995895385742, + "operator": "ge", + "actual_fraction": 0.144579177728349, + "metric": 0.8069168400660114, + "rescued": 738, + "harmed": 143, + "missed_rescue": 1968, + "avoided_harm": 568, + "selected_both_correct": 974, + "selected_both_wrong": 160 + }, + { + "budget": 0.16, + "threshold": -0.7655540704727173, + "operator": "ge", + "actual_fraction": 0.1534763578962474, + "metric": 0.8095716438257875, + "rescued": 788, + "harmed": 156, + "missed_rescue": 1918, + "avoided_harm": 555, + "selected_both_correct": 1026, + "selected_both_wrong": 169 + }, + { + "budget": 0.17, + "threshold": -0.7812193036079407, + "operator": "ge", + "actual_fraction": 0.16280404678194735, + "metric": 0.8120111932266628, + "rescued": 836, + "harmed": 170, + "missed_rescue": 1870, + "avoided_harm": 541, + "selected_both_correct": 1083, + "selected_both_wrong": 180 + }, + { + "budget": 0.18, + "threshold": -0.7954917550086975, + "operator": "ge", + "actual_fraction": 0.17306450455621727, + "metric": 0.8153117600631413, + "rescued": 891, + "harmed": 179, + "missed_rescue": 1815, + "avoided_harm": 532, + "selected_both_correct": 1151, + "selected_both_wrong": 191 + }, + { + "budget": 0.19, + "threshold": -0.8097342252731323, + "operator": "ge", + "actual_fraction": 0.18196168472411567, + "metric": 0.8172490492932482, + "rescued": 928, + "harmed": 189, + "missed_rescue": 1778, + "avoided_harm": 522, + "selected_both_correct": 1219, + "selected_both_wrong": 200 + }, + { + "budget": 0.2, + "threshold": -0.8232117891311646, + "operator": "ge", + "actual_fraction": 0.1928679055750879, + "metric": 0.8200473559589582, + "rescued": 976, + "harmed": 198, + "missed_rescue": 1730, + "avoided_harm": 513, + "selected_both_correct": 1308, + "selected_both_wrong": 206 + }, + { + "budget": 0.21, + "threshold": -0.8359466791152954, + "operator": "ge", + "actual_fraction": 0.20104757121331707, + "metric": 0.8223434024538997, + "rescued": 1015, + "harmed": 205, + "missed_rescue": 1691, + "avoided_harm": 506, + "selected_both_correct": 1369, + "selected_both_wrong": 213 + }, + { + "budget": 0.22, + "threshold": -0.8473842144012451, + "operator": "ge", + "actual_fraction": 0.21051876300495084, + "metric": 0.8249264547607089, + "rescued": 1060, + "harmed": 214, + "missed_rescue": 1646, + "avoided_harm": 497, + "selected_both_correct": 1434, + "selected_both_wrong": 226 + }, + { + "budget": 0.23, + "threshold": -0.8595863580703735, + "operator": "ge", + "actual_fraction": 0.21998995479658462, + "metric": 0.8267919925378489, + "rescued": 1097, + "harmed": 225, + "missed_rescue": 1609, + "avoided_harm": 486, + "selected_both_correct": 1510, + "selected_both_wrong": 234 + }, + { + "budget": 0.24, + "threshold": -0.8704345226287842, + "operator": "ge", + "actual_fraction": 0.22838487479371458, + "metric": 0.8283705245031212, + "rescued": 1128, + "harmed": 234, + "missed_rescue": 1578, + "avoided_harm": 477, + "selected_both_correct": 1582, + "selected_both_wrong": 239 + }, + { + "budget": 0.25, + "threshold": -0.8811548352241516, + "operator": "ge", + "actual_fraction": 0.23900408983281912, + "metric": 0.8309535768099304, + "rescued": 1174, + "harmed": 244, + "missed_rescue": 1532, + "avoided_harm": 467, + "selected_both_correct": 1665, + "selected_both_wrong": 248 + }, + { + "budget": 0.26, + "threshold": -0.8929013609886169, + "operator": "ge", + "actual_fraction": 0.2506995766664275, + "metric": 0.8342541436464088, + "rescued": 1227, + "harmed": 251, + "missed_rescue": 1479, + "avoided_harm": 460, + "selected_both_correct": 1757, + "selected_both_wrong": 259 + }, + { + "budget": 0.27, + "threshold": -0.903549313545227, + "operator": "ge", + "actual_fraction": 0.2646911099949774, + "metric": 0.8369806988591519, + "rescued": 1280, + "harmed": 266, + "missed_rescue": 1426, + "avoided_harm": 445, + "selected_both_correct": 1872, + "selected_both_wrong": 271 + }, + { + "budget": 0.28, + "threshold": -0.9115740656852722, + "operator": "ge", + "actual_fraction": 0.27437755614551196, + "metric": 0.8399225084307957, + "rescued": 1326, + "harmed": 271, + "missed_rescue": 1380, + "avoided_harm": 440, + "selected_both_correct": 1952, + "selected_both_wrong": 275 + }, + { + "budget": 0.29, + "threshold": -0.9182595014572144, + "operator": "ge", + "actual_fraction": 0.2819832101600057, + "metric": 0.8419315491138696, + "rescued": 1362, + "harmed": 279, + "missed_rescue": 1344, + "avoided_harm": 432, + "selected_both_correct": 2007, + "selected_both_wrong": 282 + }, + { + "budget": 0.3, + "threshold": -0.9258768558502197, + "operator": "ge", + "actual_fraction": 0.2907368874219703, + "metric": 0.8444428499677118, + "rescued": 1406, + "harmed": 288, + "missed_rescue": 1300, + "avoided_harm": 423, + "selected_both_correct": 2069, + "selected_both_wrong": 289 + }, + { + "budget": 0.31, + "threshold": -0.9331877827644348, + "operator": "ge", + "actual_fraction": 0.302145368443711, + "metric": 0.8466671450096864, + "rescued": 1448, + "harmed": 299, + "missed_rescue": 1258, + "avoided_harm": 412, + "selected_both_correct": 2169, + "selected_both_wrong": 295 + }, + { + "budget": 0.32, + "threshold": -0.9399217367172241, + "operator": "ge", + "actual_fraction": 0.31412786108918705, + "metric": 0.8487479371457272, + "rescued": 1492, + "harmed": 314, + "missed_rescue": 1214, + "avoided_harm": 397, + "selected_both_correct": 2272, + "selected_both_wrong": 300 + }, + { + "budget": 0.33, + "threshold": -0.9453923106193542, + "operator": "ge", + "actual_fraction": 0.3238143072397216, + "metric": 0.8506852263758341, + "rescued": 1527, + "harmed": 322, + "missed_rescue": 1179, + "avoided_harm": 389, + "selected_both_correct": 2358, + "selected_both_wrong": 306 + }, + { + "budget": 0.34, + "threshold": -0.9515207409858704, + "operator": "ge", + "actual_fraction": 0.33529453971442924, + "metric": 0.8536987874004449, + "rescued": 1578, + "harmed": 331, + "missed_rescue": 1128, + "avoided_harm": 380, + "selected_both_correct": 2453, + "selected_both_wrong": 311 + }, + { + "budget": 0.35, + "threshold": -0.9567410349845886, + "operator": "ge", + "actual_fraction": 0.3452679916768315, + "metric": 0.8557078280835186, + "rescued": 1615, + "harmed": 340, + "missed_rescue": 1091, + "avoided_harm": 371, + "selected_both_correct": 2540, + "selected_both_wrong": 317 + }, + { + "budget": 0.36, + "threshold": -0.9606627225875854, + "operator": "ge", + "actual_fraction": 0.35595895816890294, + "metric": 0.8581473774843941, + "rescued": 1655, + "harmed": 346, + "missed_rescue": 1051, + "avoided_harm": 365, + "selected_both_correct": 2634, + "selected_both_wrong": 326 + }, + { + "budget": 0.37, + "threshold": -0.9650874733924866, + "operator": "ge", + "actual_fraction": 0.36837195953218055, + "metric": 0.8605151754323025, + "rescued": 1702, + "harmed": 360, + "missed_rescue": 1004, + "avoided_harm": 351, + "selected_both_correct": 2738, + "selected_both_wrong": 334 + }, + { + "budget": 0.38, + "threshold": -0.9686882495880127, + "operator": "ge", + "actual_fraction": 0.3797086891009543, + "metric": 0.8622372103035086, + "rescued": 1741, + "harmed": 375, + "missed_rescue": 965, + "avoided_harm": 336, + "selected_both_correct": 2832, + "selected_both_wrong": 344 + }, + { + "budget": 0.39, + "threshold": -0.971785306930542, + "operator": "ge", + "actual_fraction": 0.391117170122695, + "metric": 0.8652507713281193, + "rescued": 1786, + "harmed": 378, + "missed_rescue": 920, + "avoided_harm": 333, + "selected_both_correct": 2936, + "selected_both_wrong": 351 + }, + { + "budget": 0.4, + "threshold": -0.9748144745826721, + "operator": "ge", + "actual_fraction": 0.40231039678553493, + "metric": 0.8673315634641602, + "rescued": 1820, + "harmed": 383, + "missed_rescue": 886, + "avoided_harm": 328, + "selected_both_correct": 3042, + "selected_both_wrong": 362 + }, + { + "budget": 0.41, + "threshold": -0.977255642414093, + "operator": "ge", + "actual_fraction": 0.41235560020090406, + "metric": 0.8697711128650355, + "rescued": 1863, + "harmed": 392, + "missed_rescue": 843, + "avoided_harm": 319, + "selected_both_correct": 3125, + "selected_both_wrong": 367 + }, + { + "budget": 0.42, + "threshold": -0.9794421792030334, + "operator": "ge", + "actual_fraction": 0.4231183181459425, + "metric": 0.8717801535481093, + "rescued": 1903, + "harmed": 404, + "missed_rescue": 803, + "avoided_harm": 307, + "selected_both_correct": 3218, + "selected_both_wrong": 372 + }, + { + "budget": 0.43, + "threshold": -0.981472373008728, + "operator": "ge", + "actual_fraction": 0.43172849250197315, + "metric": 0.8732151826074478, + "rescued": 1925, + "harmed": 406, + "missed_rescue": 781, + "avoided_harm": 305, + "selected_both_correct": 3309, + "selected_both_wrong": 377 + }, + { + "budget": 0.44, + "threshold": -0.9835578799247742, + "operator": "ge", + "actual_fraction": 0.4432804764296477, + "metric": 0.8749372174786539, + "rescued": 1956, + "harmed": 413, + "missed_rescue": 750, + "avoided_harm": 298, + "selected_both_correct": 3425, + "selected_both_wrong": 384 + }, + { + "budget": 0.45, + "threshold": -0.9851564168930054, + "operator": "ge", + "actual_fraction": 0.4547607089043553, + "metric": 0.8768027552557939, + "rescued": 1994, + "harmed": 425, + "missed_rescue": 712, + "avoided_harm": 286, + "selected_both_correct": 3527, + "selected_both_wrong": 392 + }, + { + "budget": 0.46, + "threshold": -0.986689031124115, + "operator": "ge", + "actual_fraction": 0.4649494152256583, + "metric": 0.8787400444859008, + "rescued": 2026, + "harmed": 430, + "missed_rescue": 680, + "avoided_harm": 281, + "selected_both_correct": 3626, + "selected_both_wrong": 398 + }, + { + "budget": 0.47, + "threshold": -0.9879001379013062, + "operator": "ge", + "actual_fraction": 0.47399009829949057, + "metric": 0.8798880677333716, + "rescued": 2047, + "harmed": 435, + "missed_rescue": 659, + "avoided_harm": 276, + "selected_both_correct": 3721, + "selected_both_wrong": 403 + }, + { + "budget": 0.48, + "threshold": -0.9891784191131592, + "operator": "ge", + "actual_fraction": 0.48446581043266124, + "metric": 0.8814665996986439, + "rescued": 2076, + "harmed": 442, + "missed_rescue": 630, + "avoided_harm": 269, + "selected_both_correct": 3824, + "selected_both_wrong": 410 + }, + { + "budget": 0.49, + "threshold": -0.990114688873291, + "operator": "ge", + "actual_fraction": 0.491210447011552, + "metric": 0.8825428714931477, + "rescued": 2095, + "harmed": 446, + "missed_rescue": 611, + "avoided_harm": 265, + "selected_both_correct": 3892, + "selected_both_wrong": 413 + }, + { + "budget": 0.5, + "threshold": -0.9910277724266052, + "operator": "ge", + "actual_fraction": 0.5008251417091196, + "metric": 0.8833321374757839, + "rescued": 2116, + "harmed": 456, + "missed_rescue": 590, + "avoided_harm": 255, + "selected_both_correct": 3984, + "selected_both_wrong": 424 + }, + { + "budget": 0.51, + "threshold": -0.9918718934059143, + "operator": "ge", + "actual_fraction": 0.5107268422185549, + "metric": 0.8839061490995193, + "rescued": 2133, + "harmed": 465, + "missed_rescue": 573, + "avoided_harm": 246, + "selected_both_correct": 4093, + "selected_both_wrong": 427 + }, + { + "budget": 0.52, + "threshold": -0.9926809668540955, + "operator": "ge", + "actual_fraction": 0.5217048145224941, + "metric": 0.88505417234699, + "rescued": 2158, + "harmed": 474, + "missed_rescue": 548, + "avoided_harm": 237, + "selected_both_correct": 4209, + "selected_both_wrong": 430 + }, + { + "budget": 0.53, + "threshold": -0.9932646751403809, + "operator": "ge", + "actual_fraction": 0.5306737461433594, + "metric": 0.8867044557652293, + "rescued": 2184, + "harmed": 477, + "missed_rescue": 522, + "avoided_harm": 234, + "selected_both_correct": 4302, + "selected_both_wrong": 433 + }, + { + "budget": 0.54, + "threshold": -0.9938417673110962, + "operator": "ge", + "actual_fraction": 0.5398579321231255, + "metric": 0.8876372246537992, + "rescued": 2203, + "harmed": 483, + "missed_rescue": 503, + "avoided_harm": 228, + "selected_both_correct": 4401, + "selected_both_wrong": 437 + }, + { + "budget": 0.55, + "threshold": -0.9944412112236023, + "operator": "ge", + "actual_fraction": 0.5504771471622301, + "metric": 0.8882112362775346, + "rescued": 2220, + "harmed": 492, + "missed_rescue": 486, + "avoided_harm": 219, + "selected_both_correct": 4514, + "selected_both_wrong": 446 + }, + { + "budget": 0.56, + "threshold": -0.9949513673782349, + "operator": "ge", + "actual_fraction": 0.5604505991246322, + "metric": 0.8893592595250054, + "rescued": 2242, + "harmed": 498, + "missed_rescue": 464, + "avoided_harm": 213, + "selected_both_correct": 4617, + "selected_both_wrong": 454 + }, + { + "budget": 0.57, + "threshold": -0.9954326152801514, + "operator": "ge", + "actual_fraction": 0.570926311257803, + "metric": 0.8905072827724761, + "rescued": 2262, + "harmed": 502, + "missed_rescue": 444, + "avoided_harm": 209, + "selected_both_correct": 4734, + "selected_both_wrong": 459 + }, + { + "budget": 0.58, + "threshold": -0.9958608746528625, + "operator": "ge", + "actual_fraction": 0.5816172777498745, + "metric": 0.8916553060199469, + "rescued": 2287, + "harmed": 511, + "missed_rescue": 419, + "avoided_harm": 200, + "selected_both_correct": 4841, + "selected_both_wrong": 467 + }, + { + "budget": 0.59, + "threshold": -0.9962128400802612, + "operator": "ge", + "actual_fraction": 0.5920929898830452, + "metric": 0.8924445720025831, + "rescued": 2308, + "harmed": 521, + "missed_rescue": 398, + "avoided_harm": 190, + "selected_both_correct": 4951, + "selected_both_wrong": 472 + }, + { + "budget": 0.6, + "threshold": -0.9965764880180359, + "operator": "ge", + "actual_fraction": 0.6034297194518189, + "metric": 0.8928750807203846, + "rescued": 2326, + "harmed": 533, + "missed_rescue": 380, + "avoided_harm": 178, + "selected_both_correct": 5075, + "selected_both_wrong": 476 + }, + { + "budget": 0.61, + "threshold": -0.9968799352645874, + "operator": "ge", + "actual_fraction": 0.6144794432087249, + "metric": 0.8936643467030208, + "rescued": 2346, + "harmed": 542, + "missed_rescue": 360, + "avoided_harm": 169, + "selected_both_correct": 5197, + "selected_both_wrong": 479 + }, + { + "budget": 0.62, + "threshold": -0.9971383810043335, + "operator": "ge", + "actual_fraction": 0.6232331204706896, + "metric": 0.8946688670445576, + "rescued": 2363, + "harmed": 545, + "missed_rescue": 343, + "avoided_harm": 166, + "selected_both_correct": 5293, + "selected_both_wrong": 485 + }, + { + "budget": 0.63, + "threshold": -0.997377872467041, + "operator": "ge", + "actual_fraction": 0.6337805840568271, + "metric": 0.8953863815742269, + "rescued": 2381, + "harmed": 553, + "missed_rescue": 325, + "avoided_harm": 158, + "selected_both_correct": 5411, + "selected_both_wrong": 488 + }, + { + "budget": 0.64, + "threshold": -0.9976021647453308, + "operator": "ge", + "actual_fraction": 0.6445433020018655, + "metric": 0.8958886417449954, + "rescued": 2398, + "harmed": 563, + "missed_rescue": 308, + "avoided_harm": 148, + "selected_both_correct": 5525, + "selected_both_wrong": 497 + }, + { + "budget": 0.65, + "threshold": -0.9978214502334595, + "operator": "ge", + "actual_fraction": 0.6565257946473416, + "metric": 0.8963909019157638, + "rescued": 2411, + "harmed": 569, + "missed_rescue": 295, + "avoided_harm": 142, + "selected_both_correct": 5668, + "selected_both_wrong": 502 + }, + { + "budget": 0.66, + "threshold": -0.9980514049530029, + "operator": "ge", + "actual_fraction": 0.6692258018224869, + "metric": 0.8972519193513668, + "rescued": 2430, + "harmed": 576, + "missed_rescue": 276, + "avoided_harm": 135, + "selected_both_correct": 5815, + "selected_both_wrong": 506 + }, + { + "budget": 0.67, + "threshold": -0.9982016086578369, + "operator": "ge", + "actual_fraction": 0.6777642247255506, + "metric": 0.8978259309751022, + "rescued": 2443, + "harmed": 581, + "missed_rescue": 263, + "avoided_harm": 130, + "selected_both_correct": 5912, + "selected_both_wrong": 510 + }, + { + "budget": 0.68, + "threshold": -0.9983693957328796, + "operator": "ge", + "actual_fraction": 0.6885269426705891, + "metric": 0.8986869484107053, + "rescued": 2462, + "harmed": 588, + "missed_rescue": 244, + "avoided_harm": 123, + "selected_both_correct": 6035, + "selected_both_wrong": 511 + }, + { + "budget": 0.69, + "threshold": -0.9984915256500244, + "operator": "ge", + "actual_fraction": 0.6977828801033221, + "metric": 0.898973954222573, + "rescued": 2473, + "harmed": 595, + "missed_rescue": 233, + "avoided_harm": 116, + "selected_both_correct": 6142, + "selected_both_wrong": 515 + }, + { + "budget": 0.7, + "threshold": -0.9986109733581543, + "operator": "gt", + "actual_fraction": 0.7064648059123197, + "metric": 0.8988304513166392, + "rescued": 2481, + "harmed": 605, + "missed_rescue": 225, + "avoided_harm": 106, + "selected_both_correct": 6242, + "selected_both_wrong": 518 + }, + { + "budget": 0.71, + "threshold": -0.9986966252326965, + "operator": "ge", + "actual_fraction": 0.7137117026619789, + "metric": 0.8996197172992753, + "rescued": 2497, + "harmed": 610, + "missed_rescue": 209, + "avoided_harm": 101, + "selected_both_correct": 6320, + "selected_both_wrong": 520 + }, + { + "budget": 0.72, + "threshold": -0.9987963438034058, + "operator": "ge", + "actual_fraction": 0.7256941953074549, + "metric": 0.9004089832819114, + "rescued": 2512, + "harmed": 614, + "missed_rescue": 194, + "avoided_harm": 97, + "selected_both_correct": 6462, + "selected_both_wrong": 526 + }, + { + "budget": 0.73, + "threshold": -0.998875081539154, + "operator": "ge", + "actual_fraction": 0.7345196240223865, + "metric": 0.9009829949056468, + "rescued": 2521, + "harmed": 615, + "missed_rescue": 185, + "avoided_harm": 96, + "selected_both_correct": 6569, + "selected_both_wrong": 532 + }, + { + "budget": 0.74, + "threshold": -0.9989551305770874, + "operator": "ge", + "actual_fraction": 0.745067087608524, + "metric": 0.901772260888283, + "rescued": 2535, + "harmed": 618, + "missed_rescue": 171, + "avoided_harm": 93, + "selected_both_correct": 6698, + "selected_both_wrong": 533 + }, + { + "budget": 0.75, + "threshold": -0.9990480542182922, + "operator": "ge", + "actual_fraction": 0.7568343258950994, + "metric": 0.9023462725120184, + "rescued": 2548, + "harmed": 623, + "missed_rescue": 158, + "avoided_harm": 88, + "selected_both_correct": 6839, + "selected_both_wrong": 538 + }, + { + "budget": 0.76, + "threshold": -0.999126136302948, + "operator": "ge", + "actual_fraction": 0.7663772691397001, + "metric": 0.9028485326827869, + "rescued": 2561, + "harmed": 629, + "missed_rescue": 145, + "avoided_harm": 82, + "selected_both_correct": 6951, + "selected_both_wrong": 540 + }, + { + "budget": 0.77, + "threshold": -0.999184787273407, + "operator": "gt", + "actual_fraction": 0.7746286862308962, + "metric": 0.9030637870416875, + "rescued": 2567, + "harmed": 632, + "missed_rescue": 139, + "avoided_harm": 79, + "selected_both_correct": 7054, + "selected_both_wrong": 543 + }, + { + "budget": 0.78, + "threshold": -0.9992268085479736, + "operator": "ge", + "actual_fraction": 0.7817320800746215, + "metric": 0.9028485326827869, + "rescued": 2570, + "harmed": 638, + "missed_rescue": 136, + "avoided_harm": 73, + "selected_both_correct": 7143, + "selected_both_wrong": 544 + }, + { + "budget": 0.79, + "threshold": -0.9992831349372864, + "operator": "ge", + "actual_fraction": 0.7928535552844945, + "metric": 0.9028485326827869, + "rescued": 2579, + "harmed": 647, + "missed_rescue": 127, + "avoided_harm": 64, + "selected_both_correct": 7279, + "selected_both_wrong": 545 + }, + { + "budget": 0.8, + "threshold": -0.9993318915367126, + "operator": "ge", + "actual_fraction": 0.8023247470761283, + "metric": 0.9028485326827869, + "rescued": 2581, + "harmed": 649, + "missed_rescue": 125, + "avoided_harm": 62, + "selected_both_correct": 7406, + "selected_both_wrong": 546 + }, + { + "budget": 0.81, + "threshold": -0.9993773102760315, + "operator": "ge", + "actual_fraction": 0.811795938867762, + "metric": 0.9027767812298199, + "rescued": 2588, + "harmed": 657, + "missed_rescue": 118, + "avoided_harm": 54, + "selected_both_correct": 7521, + "selected_both_wrong": 548 + }, + { + "budget": 0.82, + "threshold": -0.9994175434112549, + "operator": "ge", + "actual_fraction": 0.8214106335653297, + "metric": 0.9034225443065222, + "rescued": 2601, + "harmed": 661, + "missed_rescue": 105, + "avoided_harm": 50, + "selected_both_correct": 7637, + "selected_both_wrong": 549 + }, + { + "budget": 0.83, + "threshold": -0.999452531337738, + "operator": "ge", + "actual_fraction": 0.8298773050154266, + "metric": 0.9037095501183899, + "rescued": 2607, + "harmed": 663, + "missed_rescue": 99, + "avoided_harm": 48, + "selected_both_correct": 7745, + "selected_both_wrong": 551 + }, + { + "budget": 0.84, + "threshold": -0.9994945526123047, + "operator": "ge", + "actual_fraction": 0.8420750520198034, + "metric": 0.9037095501183899, + "rescued": 2612, + "harmed": 668, + "missed_rescue": 94, + "avoided_harm": 43, + "selected_both_correct": 7903, + "selected_both_wrong": 553 + }, + { + "budget": 0.85, + "threshold": -0.9995274543762207, + "operator": "gt", + "actual_fraction": 0.8502547176580326, + "metric": 0.9039248044772906, + "rescued": 2616, + "harmed": 669, + "missed_rescue": 90, + "avoided_harm": 42, + "selected_both_correct": 8008, + "selected_both_wrong": 557 + }, + { + "budget": 0.86, + "threshold": -0.9995617270469666, + "operator": "ge", + "actual_fraction": 0.8604434239793356, + "metric": 0.9042835617421253, + "rescued": 2623, + "harmed": 671, + "missed_rescue": 83, + "avoided_harm": 40, + "selected_both_correct": 8140, + "selected_both_wrong": 558 + }, + { + "budget": 0.87, + "threshold": -0.9995949864387512, + "operator": "ge", + "actual_fraction": 0.8690535983353663, + "metric": 0.904570567553993, + "rescued": 2629, + "harmed": 673, + "missed_rescue": 77, + "avoided_harm": 38, + "selected_both_correct": 8251, + "selected_both_wrong": 559 + }, + { + "budget": 0.88, + "threshold": -0.9996248483657837, + "operator": "ge", + "actual_fraction": 0.8778790270502977, + "metric": 0.904498816101026, + "rescued": 2634, + "harmed": 679, + "missed_rescue": 72, + "avoided_harm": 32, + "selected_both_correct": 8361, + "selected_both_wrong": 561 + }, + { + "budget": 0.89, + "threshold": -0.9996511936187744, + "operator": "ge", + "actual_fraction": 0.886058692688527, + "metric": 0.9050010762717945, + "rescued": 2642, + "harmed": 680, + "missed_rescue": 64, + "avoided_harm": 31, + "selected_both_correct": 8464, + "selected_both_wrong": 563 + }, + { + "budget": 0.9, + "threshold": -0.9996830224990845, + "operator": "ge", + "actual_fraction": 0.8961756475568631, + "metric": 0.9052163306306953, + "rescued": 2648, + "harmed": 683, + "missed_rescue": 58, + "avoided_harm": 28, + "selected_both_correct": 8590, + "selected_both_wrong": 569 + }, + { + "budget": 0.91, + "threshold": -0.9997097849845886, + "operator": "gt", + "actual_fraction": 0.9053598335366291, + "metric": 0.9055750878955299, + "rescued": 2654, + "harmed": 684, + "missed_rescue": 52, + "avoided_harm": 27, + "selected_both_correct": 8711, + "selected_both_wrong": 569 + }, + { + "budget": 0.92, + "threshold": -0.9997391104698181, + "operator": "ge", + "actual_fraction": 0.9164813087465021, + "metric": 0.9057185908014638, + "rescued": 2659, + "harmed": 687, + "missed_rescue": 47, + "avoided_harm": 24, + "selected_both_correct": 8855, + "selected_both_wrong": 572 + }, + { + "budget": 0.93, + "threshold": -0.9997641444206238, + "operator": "ge", + "actual_fraction": 0.9265265121618713, + "metric": 0.9060773480662984, + "rescued": 2667, + "harmed": 690, + "missed_rescue": 39, + "avoided_harm": 21, + "selected_both_correct": 8984, + "selected_both_wrong": 572 + }, + { + "budget": 0.94, + "threshold": -0.9997876286506653, + "operator": "ge", + "actual_fraction": 0.9362847097653727, + "metric": 0.906364353878166, + "rescued": 2674, + "harmed": 693, + "missed_rescue": 32, + "avoided_harm": 18, + "selected_both_correct": 9109, + "selected_both_wrong": 573 + }, + { + "budget": 0.95, + "threshold": -0.9998131394386292, + "operator": "ge", + "actual_fraction": 0.9475496878811795, + "metric": 0.9065078567840998, + "rescued": 2680, + "harmed": 697, + "missed_rescue": 26, + "avoided_harm": 14, + "selected_both_correct": 9256, + "selected_both_wrong": 573 + }, + { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9593169261677549, + "metric": 0.906436105331133, + "rescued": 2682, + "harmed": 700, + "missed_rescue": 24, + "avoided_harm": 11, + "selected_both_correct": 9413, + "selected_both_wrong": 575 + }, + { + "budget": 0.97, + "threshold": -0.999868631362915, + "operator": "ge", + "actual_fraction": 0.9700796441127932, + "metric": 0.9070101169548683, + "rescued": 2690, + "harmed": 700, + "missed_rescue": 16, + "avoided_harm": 11, + "selected_both_correct": 9554, + "selected_both_wrong": 576 + }, + { + "budget": 0.98, + "threshold": -0.9998925924301147, + "operator": "ge", + "actual_fraction": 0.9793355815455262, + "metric": 0.9071536198608022, + "rescued": 2696, + "harmed": 704, + "missed_rescue": 10, + "avoided_harm": 7, + "selected_both_correct": 9672, + "selected_both_wrong": 577 + }, + { + "budget": 0.99, + "threshold": -0.99992835521698, + "operator": "ge", + "actual_fraction": 0.9909593169261678, + "metric": 0.9071536198608022, + "rescued": 2702, + "harmed": 710, + "missed_rescue": 4, + "avoided_harm": 1, + "selected_both_correct": 9821, + "selected_both_wrong": 578 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9073688742197029, + "rescued": 2706, + "harmed": 711, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9940, + "selected_both_wrong": 580 + } + ] + }, + "learned": { + "selected_budget_index": 99, + "validation_area_0_25": 0.7934259524491549, + "validation_point": { + "budget": 0.99, + "threshold": -4.5110697746276855, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.898166714408479, + "rescued": 1381, + "harmed": 386, + "missed_rescue": 8, + "avoided_harm": 2, + "selected_both_correct": 4833, + "selected_both_wrong": 312 + }, + "test_point": { + "budget": 0.99, + "threshold": -4.5110697746276855, + "operator": "ge", + "actual_fraction": 0.9896677907727631, + "metric": 0.9070818684078352, + "rescued": 2695, + "harmed": 704, + "missed_rescue": 11, + "avoided_harm": 7, + "selected_both_correct": 9819, + "selected_both_wrong": 575 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7556574047550845, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1389, + "avoided_harm": 388, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.4486454725265503, + "operator": "ge", + "actual_fraction": 0.009882555141793183, + "metric": 0.7595244915496993, + "rescued": 28, + "harmed": 1, + "missed_rescue": 1361, + "avoided_harm": 387, + "selected_both_correct": 36, + "selected_both_wrong": 4 + }, + { + "budget": 0.02, + "threshold": 1.3545215129852295, + "operator": "ge", + "actual_fraction": 0.01990833572042395, + "metric": 0.7632483529074764, + "rescued": 59, + "harmed": 6, + "missed_rescue": 1330, + "avoided_harm": 382, + "selected_both_correct": 65, + "selected_both_wrong": 9 + }, + { + "budget": 0.03, + "threshold": 1.2635189294815063, + "operator": "ge", + "actual_fraction": 0.029934116299054712, + "metric": 0.7665425379547408, + "rescued": 86, + "harmed": 10, + "missed_rescue": 1303, + "avoided_harm": 378, + "selected_both_correct": 99, + "selected_both_wrong": 14 + }, + { + "budget": 0.04, + "threshold": 1.178093433380127, + "operator": "ge", + "actual_fraction": 0.039959896877685475, + "metric": 0.7694070466914924, + "rescued": 116, + "harmed": 20, + "missed_rescue": 1273, + "avoided_harm": 368, + "selected_both_correct": 127, + "selected_both_wrong": 16 + }, + { + "budget": 0.05, + "threshold": 1.101384162902832, + "operator": "ge", + "actual_fraction": 0.04998567745631624, + "metric": 0.7721283299914065, + "rescued": 138, + "harmed": 23, + "missed_rescue": 1251, + "avoided_harm": 365, + "selected_both_correct": 168, + "selected_both_wrong": 20 + }, + { + "budget": 0.06, + "threshold": 1.0296903848648071, + "operator": "ge", + "actual_fraction": 0.059868232598109423, + "metric": 0.7741334861071326, + "rescued": 159, + "harmed": 30, + "missed_rescue": 1230, + "avoided_harm": 358, + "selected_both_correct": 204, + "selected_both_wrong": 25 + }, + { + "budget": 0.07, + "threshold": 0.9584200978279114, + "operator": "ge", + "actual_fraction": 0.06989401317674018, + "metric": 0.777427671154397, + "rescued": 186, + "harmed": 34, + "missed_rescue": 1203, + "avoided_harm": 354, + "selected_both_correct": 240, + "selected_both_wrong": 28 + }, + { + "budget": 0.08, + "threshold": 0.9015151262283325, + "operator": "ge", + "actual_fraction": 0.07991979375537095, + "metric": 0.780865081638499, + "rescued": 216, + "harmed": 40, + "missed_rescue": 1173, + "avoided_harm": 348, + "selected_both_correct": 269, + "selected_both_wrong": 33 + }, + { + "budget": 0.09, + "threshold": 0.8391129970550537, + "operator": "ge", + "actual_fraction": 0.08994557433400172, + "metric": 0.7841592666857634, + "rescued": 247, + "harmed": 48, + "missed_rescue": 1142, + "avoided_harm": 340, + "selected_both_correct": 297, + "selected_both_wrong": 36 + }, + { + "budget": 0.1, + "threshold": 0.7945014238357544, + "operator": "ge", + "actual_fraction": 0.09997135491263248, + "metric": 0.7877399026067029, + "rescued": 277, + "harmed": 53, + "missed_rescue": 1112, + "avoided_harm": 335, + "selected_both_correct": 326, + "selected_both_wrong": 42 + }, + { + "budget": 0.11, + "threshold": 0.7454464435577393, + "operator": "ge", + "actual_fraction": 0.10999713549126325, + "metric": 0.789458607848754, + "rescued": 297, + "harmed": 61, + "missed_rescue": 1092, + "avoided_harm": 327, + "selected_both_correct": 363, + "selected_both_wrong": 47 + }, + { + "budget": 0.12, + "threshold": 0.7009376883506775, + "operator": "ge", + "actual_fraction": 0.11987969063305644, + "metric": 0.7923231165855056, + "rescued": 322, + "harmed": 66, + "missed_rescue": 1067, + "avoided_harm": 322, + "selected_both_correct": 396, + "selected_both_wrong": 53 + }, + { + "budget": 0.13, + "threshold": 0.651968240737915, + "operator": "ge", + "actual_fraction": 0.1299054712116872, + "metric": 0.7964766542537954, + "rescued": 352, + "harmed": 67, + "missed_rescue": 1037, + "avoided_harm": 321, + "selected_both_correct": 433, + "selected_both_wrong": 55 + }, + { + "budget": 0.14, + "threshold": 0.5855442881584167, + "operator": "ge", + "actual_fraction": 0.13993125179031796, + "metric": 0.799054712116872, + "rescued": 378, + "harmed": 75, + "missed_rescue": 1011, + "avoided_harm": 313, + "selected_both_correct": 467, + "selected_both_wrong": 57 + }, + { + "budget": 0.15, + "threshold": 0.5396223068237305, + "operator": "ge", + "actual_fraction": 0.14995703236894872, + "metric": 0.8016327699799485, + "rescued": 400, + "harmed": 79, + "missed_rescue": 989, + "avoided_harm": 309, + "selected_both_correct": 507, + "selected_both_wrong": 61 + }, + { + "budget": 0.16, + "threshold": 0.4906216263771057, + "operator": "ge", + "actual_fraction": 0.1599828129475795, + "metric": 0.8044972787167001, + "rescued": 424, + "harmed": 83, + "missed_rescue": 965, + "avoided_harm": 305, + "selected_both_correct": 541, + "selected_both_wrong": 69 + }, + { + "budget": 0.17, + "threshold": 0.44199177622795105, + "operator": "ge", + "actual_fraction": 0.16986536808937266, + "metric": 0.8072185620166141, + "rescued": 451, + "harmed": 91, + "missed_rescue": 938, + "avoided_harm": 297, + "selected_both_correct": 572, + "selected_both_wrong": 72 + }, + { + "budget": 0.18, + "threshold": 0.3899296522140503, + "operator": "ge", + "actual_fraction": 0.17989114866800343, + "metric": 0.8090804926955028, + "rescued": 470, + "harmed": 97, + "missed_rescue": 919, + "avoided_harm": 291, + "selected_both_correct": 611, + "selected_both_wrong": 78 + }, + { + "budget": 0.19, + "threshold": 0.325008362531662, + "operator": "ge", + "actual_fraction": 0.1899169292466342, + "metric": 0.8115153251217416, + "rescued": 497, + "harmed": 107, + "missed_rescue": 892, + "avoided_harm": 281, + "selected_both_correct": 642, + "selected_both_wrong": 80 + }, + { + "budget": 0.2, + "threshold": 0.26310965418815613, + "operator": "ge", + "actual_fraction": 0.19994270982526496, + "metric": 0.8149527356058436, + "rescued": 528, + "harmed": 114, + "missed_rescue": 861, + "avoided_harm": 274, + "selected_both_correct": 668, + "selected_both_wrong": 86 + }, + { + "budget": 0.21, + "threshold": 0.19935977458953857, + "operator": "ge", + "actual_fraction": 0.20996849040389573, + "metric": 0.8173875680320825, + "rescued": 550, + "harmed": 119, + "missed_rescue": 839, + "avoided_harm": 269, + "selected_both_correct": 705, + "selected_both_wrong": 92 + }, + { + "budget": 0.22, + "threshold": 0.13677698373794556, + "operator": "ge", + "actual_fraction": 0.2199942709825265, + "metric": 0.8183901460899455, + "rescued": 565, + "harmed": 127, + "missed_rescue": 824, + "avoided_harm": 261, + "selected_both_correct": 740, + "selected_both_wrong": 104 + }, + { + "budget": 0.23, + "threshold": 0.0732785016298294, + "operator": "ge", + "actual_fraction": 0.22987682612431967, + "metric": 0.82168433113721, + "rescued": 591, + "harmed": 130, + "missed_rescue": 798, + "avoided_harm": 258, + "selected_both_correct": 777, + "selected_both_wrong": 107 + }, + { + "budget": 0.24, + "threshold": 0.008367598056793213, + "operator": "ge", + "actual_fraction": 0.23990260670295044, + "metric": 0.8228301346319107, + "rescued": 606, + "harmed": 137, + "missed_rescue": 783, + "avoided_harm": 251, + "selected_both_correct": 818, + "selected_both_wrong": 114 + }, + { + "budget": 0.25, + "threshold": -0.0719703733921051, + "operator": "ge", + "actual_fraction": 0.2499283872815812, + "metric": 0.8256946433686623, + "rescued": 629, + "harmed": 140, + "missed_rescue": 760, + "avoided_harm": 248, + "selected_both_correct": 861, + "selected_both_wrong": 115 + }, + { + "budget": 0.26, + "threshold": -0.14035016298294067, + "operator": "ge", + "actual_fraction": 0.25995416786021197, + "metric": 0.8272701231738757, + "rescued": 647, + "harmed": 147, + "missed_rescue": 742, + "avoided_harm": 241, + "selected_both_correct": 898, + "selected_both_wrong": 123 + }, + { + "budget": 0.27, + "threshold": -0.2074521780014038, + "operator": "ge", + "actual_fraction": 0.26997994843884277, + "metric": 0.8291320538527642, + "rescued": 665, + "harmed": 152, + "missed_rescue": 724, + "avoided_harm": 236, + "selected_both_correct": 939, + "selected_both_wrong": 129 + }, + { + "budget": 0.28, + "threshold": -0.274451345205307, + "operator": "ge", + "actual_fraction": 0.2798625035806359, + "metric": 0.8299914064737898, + "rescued": 677, + "harmed": 158, + "missed_rescue": 712, + "avoided_harm": 230, + "selected_both_correct": 985, + "selected_both_wrong": 134 + }, + { + "budget": 0.29, + "threshold": -0.331489622592926, + "operator": "ge", + "actual_fraction": 0.2898882841592667, + "metric": 0.8324262389000286, + "rescued": 698, + "harmed": 162, + "missed_rescue": 691, + "avoided_harm": 226, + "selected_both_correct": 1023, + "selected_both_wrong": 141 + }, + { + "budget": 0.3, + "threshold": -0.4019598960876465, + "operator": "ge", + "actual_fraction": 0.29991406473789745, + "metric": 0.8344313950157548, + "rescued": 717, + "harmed": 167, + "missed_rescue": 672, + "avoided_harm": 221, + "selected_both_correct": 1065, + "selected_both_wrong": 145 + }, + { + "budget": 0.31, + "threshold": -0.4725560247898102, + "operator": "ge", + "actual_fraction": 0.30993984531652824, + "metric": 0.8367230020051561, + "rescued": 738, + "harmed": 172, + "missed_rescue": 651, + "avoided_harm": 216, + "selected_both_correct": 1105, + "selected_both_wrong": 149 + }, + { + "budget": 0.32, + "threshold": -0.5430232882499695, + "operator": "ge", + "actual_fraction": 0.319965625895159, + "metric": 0.8390146089945575, + "rescued": 758, + "harmed": 176, + "missed_rescue": 631, + "avoided_harm": 212, + "selected_both_correct": 1145, + "selected_both_wrong": 155 + }, + { + "budget": 0.33, + "threshold": -0.6012662649154663, + "operator": "ge", + "actual_fraction": 0.3299914064737898, + "metric": 0.8413062159839587, + "rescued": 778, + "harmed": 180, + "missed_rescue": 611, + "avoided_harm": 208, + "selected_both_correct": 1188, + "selected_both_wrong": 158 + }, + { + "budget": 0.34, + "threshold": -0.6759253740310669, + "operator": "ge", + "actual_fraction": 0.3398739616155829, + "metric": 0.844313950157548, + "rescued": 802, + "harmed": 183, + "missed_rescue": 587, + "avoided_harm": 205, + "selected_both_correct": 1225, + "selected_both_wrong": 163 + }, + { + "budget": 0.35, + "threshold": -0.7336556911468506, + "operator": "ge", + "actual_fraction": 0.3498997421942137, + "metric": 0.847035233457462, + "rescued": 824, + "harmed": 186, + "missed_rescue": 565, + "avoided_harm": 202, + "selected_both_correct": 1263, + "selected_both_wrong": 170 + }, + { + "budget": 0.36, + "threshold": -0.7973066568374634, + "operator": "ge", + "actual_fraction": 0.35992552277284445, + "metric": 0.8504726439415641, + "rescued": 849, + "harmed": 187, + "missed_rescue": 540, + "avoided_harm": 201, + "selected_both_correct": 1305, + "selected_both_wrong": 172 + }, + { + "budget": 0.37, + "threshold": -0.86806321144104, + "operator": "ge", + "actual_fraction": 0.36995130335147525, + "metric": 0.8526210254941278, + "rescued": 868, + "harmed": 191, + "missed_rescue": 521, + "avoided_harm": 197, + "selected_both_correct": 1349, + "selected_both_wrong": 175 + }, + { + "budget": 0.38, + "threshold": -0.9553278684616089, + "operator": "ge", + "actual_fraction": 0.379977083930106, + "metric": 0.8547694070466915, + "rescued": 886, + "harmed": 194, + "missed_rescue": 503, + "avoided_harm": 194, + "selected_both_correct": 1395, + "selected_both_wrong": 178 + }, + { + "budget": 0.39, + "threshold": -1.0301940441131592, + "operator": "ge", + "actual_fraction": 0.3898596390718992, + "metric": 0.8570610140360928, + "rescued": 904, + "harmed": 196, + "missed_rescue": 485, + "avoided_harm": 192, + "selected_both_correct": 1441, + "selected_both_wrong": 181 + }, + { + "budget": 0.4, + "threshold": -1.1348119974136353, + "operator": "ge", + "actual_fraction": 0.3998854196505299, + "metric": 0.8592093955886565, + "rescued": 923, + "harmed": 200, + "missed_rescue": 466, + "avoided_harm": 188, + "selected_both_correct": 1481, + "selected_both_wrong": 188 + }, + { + "budget": 0.41, + "threshold": -1.226643681526184, + "operator": "ge", + "actual_fraction": 0.4099112002291607, + "metric": 0.8609281008307076, + "rescued": 938, + "harmed": 203, + "missed_rescue": 451, + "avoided_harm": 185, + "selected_both_correct": 1530, + "selected_both_wrong": 191 + }, + { + "budget": 0.42, + "threshold": -1.2974129915237427, + "operator": "ge", + "actual_fraction": 0.41993698080779146, + "metric": 0.8619306788885706, + "rescued": 950, + "harmed": 208, + "missed_rescue": 439, + "avoided_harm": 180, + "selected_both_correct": 1576, + "selected_both_wrong": 198 + }, + { + "budget": 0.43, + "threshold": -1.4153810739517212, + "operator": "ge", + "actual_fraction": 0.42996276138642225, + "metric": 0.8637926095674592, + "rescued": 966, + "harmed": 211, + "missed_rescue": 423, + "avoided_harm": 177, + "selected_both_correct": 1625, + "selected_both_wrong": 200 + }, + { + "budget": 0.44, + "threshold": -1.5353657007217407, + "operator": "ge", + "actual_fraction": 0.439988541965053, + "metric": 0.8655113148095102, + "rescued": 982, + "harmed": 215, + "missed_rescue": 407, + "avoided_harm": 173, + "selected_both_correct": 1671, + "selected_both_wrong": 204 + }, + { + "budget": 0.45, + "threshold": -1.6688950061798096, + "operator": "ge", + "actual_fraction": 0.4498710971068462, + "metric": 0.8663706674305357, + "rescued": 993, + "harmed": 220, + "missed_rescue": 396, + "avoided_harm": 168, + "selected_both_correct": 1717, + "selected_both_wrong": 211 + }, + { + "budget": 0.46, + "threshold": -1.7557787895202637, + "operator": "ge", + "actual_fraction": 0.45989687768547693, + "metric": 0.8670867946147236, + "rescued": 1004, + "harmed": 226, + "missed_rescue": 385, + "avoided_harm": 162, + "selected_both_correct": 1768, + "selected_both_wrong": 213 + }, + { + "budget": 0.47, + "threshold": -1.841616153717041, + "operator": "ge", + "actual_fraction": 0.46992265826410773, + "metric": 0.8680893726725867, + "rescued": 1014, + "harmed": 229, + "missed_rescue": 375, + "avoided_harm": 159, + "selected_both_correct": 1825, + "selected_both_wrong": 213 + }, + { + "budget": 0.48, + "threshold": -1.9284571409225464, + "operator": "ge", + "actual_fraction": 0.47994843884273847, + "metric": 0.8690919507304498, + "rescued": 1024, + "harmed": 232, + "missed_rescue": 365, + "avoided_harm": 156, + "selected_both_correct": 1880, + "selected_both_wrong": 215 + }, + { + "budget": 0.49, + "threshold": -2.0005507469177246, + "operator": "ge", + "actual_fraction": 0.48997421942136926, + "metric": 0.8693784016041249, + "rescued": 1032, + "harmed": 238, + "missed_rescue": 357, + "avoided_harm": 150, + "selected_both_correct": 1932, + "selected_both_wrong": 219 + }, + { + "budget": 0.5, + "threshold": -2.0627641677856445, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.8702377542251504, + "rescued": 1042, + "harmed": 242, + "missed_rescue": 347, + "avoided_harm": 146, + "selected_both_correct": 1988, + "selected_both_wrong": 219 + }, + { + "budget": 0.51, + "threshold": -2.1383652687072754, + "operator": "ge", + "actual_fraction": 0.5098825551417931, + "metric": 0.8709538814093383, + "rescued": 1049, + "harmed": 244, + "missed_rescue": 340, + "avoided_harm": 144, + "selected_both_correct": 2047, + "selected_both_wrong": 220 + }, + { + "budget": 0.52, + "threshold": -2.2056591510772705, + "operator": "ge", + "actual_fraction": 0.5199083357204239, + "metric": 0.8718132340303638, + "rescued": 1058, + "harmed": 247, + "missed_rescue": 331, + "avoided_harm": 141, + "selected_both_correct": 2104, + "selected_both_wrong": 221 + }, + { + "budget": 0.53, + "threshold": -2.2718443870544434, + "operator": "ge", + "actual_fraction": 0.5299341162990547, + "metric": 0.8718132340303638, + "rescued": 1062, + "harmed": 251, + "missed_rescue": 327, + "avoided_harm": 137, + "selected_both_correct": 2161, + "selected_both_wrong": 226 + }, + { + "budget": 0.54, + "threshold": -2.325044631958008, + "operator": "ge", + "actual_fraction": 0.5399598968776855, + "metric": 0.8719564594672014, + "rescued": 1067, + "harmed": 255, + "missed_rescue": 322, + "avoided_harm": 133, + "selected_both_correct": 2221, + "selected_both_wrong": 227 + }, + { + "budget": 0.55, + "threshold": -2.380568504333496, + "operator": "ge", + "actual_fraction": 0.5499856774563162, + "metric": 0.8729590375250644, + "rescued": 1075, + "harmed": 256, + "missed_rescue": 314, + "avoided_harm": 132, + "selected_both_correct": 2280, + "selected_both_wrong": 229 + }, + { + "budget": 0.56, + "threshold": -2.431140184402466, + "operator": "ge", + "actual_fraction": 0.5598682325981095, + "metric": 0.8735319392724148, + "rescued": 1080, + "harmed": 257, + "missed_rescue": 309, + "avoided_harm": 131, + "selected_both_correct": 2340, + "selected_both_wrong": 232 + }, + { + "budget": 0.57, + "threshold": -2.4825475215911865, + "operator": "ge", + "actual_fraction": 0.5698940131767402, + "metric": 0.8733887138355771, + "rescued": 1084, + "harmed": 262, + "missed_rescue": 305, + "avoided_harm": 126, + "selected_both_correct": 2400, + "selected_both_wrong": 233 + }, + { + "budget": 0.58, + "threshold": -2.532240867614746, + "operator": "ge", + "actual_fraction": 0.579919793755371, + "metric": 0.8733887138355771, + "rescued": 1088, + "harmed": 266, + "missed_rescue": 301, + "avoided_harm": 122, + "selected_both_correct": 2460, + "selected_both_wrong": 235 + }, + { + "budget": 0.59, + "threshold": -2.575655460357666, + "operator": "ge", + "actual_fraction": 0.5899455743340017, + "metric": 0.8735319392724148, + "rescued": 1091, + "harmed": 268, + "missed_rescue": 298, + "avoided_harm": 120, + "selected_both_correct": 2521, + "selected_both_wrong": 239 + }, + { + "budget": 0.6, + "threshold": -2.627545118331909, + "operator": "ge", + "actual_fraction": 0.5999713549126325, + "metric": 0.8735319392724148, + "rescued": 1095, + "harmed": 272, + "missed_rescue": 294, + "avoided_harm": 116, + "selected_both_correct": 2578, + "selected_both_wrong": 244 + }, + { + "budget": 0.61, + "threshold": -2.667684316635132, + "operator": "ge", + "actual_fraction": 0.6099971354912632, + "metric": 0.8739616155829275, + "rescued": 1099, + "harmed": 273, + "missed_rescue": 290, + "avoided_harm": 115, + "selected_both_correct": 2639, + "selected_both_wrong": 248 + }, + { + "budget": 0.62, + "threshold": -2.7069380283355713, + "operator": "ge", + "actual_fraction": 0.6198796906330565, + "metric": 0.8749641936407906, + "rescued": 1109, + "harmed": 276, + "missed_rescue": 280, + "avoided_harm": 112, + "selected_both_correct": 2692, + "selected_both_wrong": 251 + }, + { + "budget": 0.63, + "threshold": -2.7543530464172363, + "operator": "ge", + "actual_fraction": 0.6299054712116872, + "metric": 0.8746777427671154, + "rescued": 1111, + "harmed": 280, + "missed_rescue": 278, + "avoided_harm": 108, + "selected_both_correct": 2754, + "selected_both_wrong": 253 + }, + { + "budget": 0.64, + "threshold": -2.797299861907959, + "operator": "ge", + "actual_fraction": 0.639931251790318, + "metric": 0.8759667716986537, + "rescued": 1122, + "harmed": 282, + "missed_rescue": 267, + "avoided_harm": 106, + "selected_both_correct": 2811, + "selected_both_wrong": 253 + }, + { + "budget": 0.65, + "threshold": -2.84092378616333, + "operator": "ge", + "actual_fraction": 0.6499570323689488, + "metric": 0.876539673446004, + "rescued": 1130, + "harmed": 286, + "missed_rescue": 259, + "avoided_harm": 102, + "selected_both_correct": 2868, + "selected_both_wrong": 254 + }, + { + "budget": 0.66, + "threshold": -2.878750801086426, + "operator": "ge", + "actual_fraction": 0.6599828129475795, + "metric": 0.8772558006301919, + "rescued": 1138, + "harmed": 289, + "missed_rescue": 251, + "avoided_harm": 99, + "selected_both_correct": 2927, + "selected_both_wrong": 254 + }, + { + "budget": 0.67, + "threshold": -2.9207773208618164, + "operator": "ge", + "actual_fraction": 0.6698653680893727, + "metric": 0.8781151532512174, + "rescued": 1146, + "harmed": 291, + "missed_rescue": 243, + "avoided_harm": 97, + "selected_both_correct": 2986, + "selected_both_wrong": 254 + }, + { + "budget": 0.68, + "threshold": -2.963459014892578, + "operator": "ge", + "actual_fraction": 0.6798911486680035, + "metric": 0.8786880549985677, + "rescued": 1153, + "harmed": 294, + "missed_rescue": 236, + "avoided_harm": 94, + "selected_both_correct": 3044, + "selected_both_wrong": 256 + }, + { + "budget": 0.69, + "threshold": -3.001129627227783, + "operator": "ge", + "actual_fraction": 0.6899169292466342, + "metric": 0.8796906330564308, + "rescued": 1163, + "harmed": 297, + "missed_rescue": 226, + "avoided_harm": 91, + "selected_both_correct": 3099, + "selected_both_wrong": 258 + }, + { + "budget": 0.7, + "threshold": -3.040706157684326, + "operator": "ge", + "actual_fraction": 0.699942709825265, + "metric": 0.8802635348037812, + "rescued": 1171, + "harmed": 301, + "missed_rescue": 218, + "avoided_harm": 87, + "selected_both_correct": 3156, + "selected_both_wrong": 259 + }, + { + "budget": 0.71, + "threshold": -3.0831708908081055, + "operator": "ge", + "actual_fraction": 0.7099684904038958, + "metric": 0.8815525637353194, + "rescued": 1181, + "harmed": 302, + "missed_rescue": 208, + "avoided_harm": 86, + "selected_both_correct": 3211, + "selected_both_wrong": 263 + }, + { + "budget": 0.72, + "threshold": -3.121913433074951, + "operator": "ge", + "actual_fraction": 0.7199942709825266, + "metric": 0.88269836723002, + "rescued": 1192, + "harmed": 305, + "missed_rescue": 197, + "avoided_harm": 83, + "selected_both_correct": 3267, + "selected_both_wrong": 263 + }, + { + "budget": 0.73, + "threshold": -3.1643247604370117, + "operator": "ge", + "actual_fraction": 0.7298768261243197, + "metric": 0.8832712689773704, + "rescued": 1198, + "harmed": 307, + "missed_rescue": 191, + "avoided_harm": 81, + "selected_both_correct": 3327, + "selected_both_wrong": 264 + }, + { + "budget": 0.74, + "threshold": -3.1954593658447266, + "operator": "ge", + "actual_fraction": 0.7399026067029505, + "metric": 0.8831280435405328, + "rescued": 1200, + "harmed": 310, + "missed_rescue": 189, + "avoided_harm": 78, + "selected_both_correct": 3388, + "selected_both_wrong": 268 + }, + { + "budget": 0.75, + "threshold": -3.230159282684326, + "operator": "ge", + "actual_fraction": 0.7499283872815812, + "metric": 0.8847035233457462, + "rescued": 1211, + "harmed": 310, + "missed_rescue": 178, + "avoided_harm": 78, + "selected_both_correct": 3446, + "selected_both_wrong": 269 + }, + { + "budget": 0.76, + "threshold": -3.2606701850891113, + "operator": "ge", + "actual_fraction": 0.759954167860212, + "metric": 0.8851331996562589, + "rescued": 1218, + "harmed": 314, + "missed_rescue": 171, + "avoided_harm": 74, + "selected_both_correct": 3505, + "selected_both_wrong": 269 + }, + { + "budget": 0.77, + "threshold": -3.3036723136901855, + "operator": "ge", + "actual_fraction": 0.7699799484388428, + "metric": 0.8862790031509596, + "rescued": 1229, + "harmed": 317, + "missed_rescue": 160, + "avoided_harm": 71, + "selected_both_correct": 3559, + "selected_both_wrong": 271 + }, + { + "budget": 0.78, + "threshold": -3.3321590423583984, + "operator": "ge", + "actual_fraction": 0.7798625035806359, + "metric": 0.8868519048983099, + "rescued": 1238, + "harmed": 322, + "missed_rescue": 151, + "avoided_harm": 66, + "selected_both_correct": 3611, + "selected_both_wrong": 274 + }, + { + "budget": 0.79, + "threshold": -3.368250846862793, + "operator": "ge", + "actual_fraction": 0.7898882841592667, + "metric": 0.8871383557719851, + "rescued": 1243, + "harmed": 325, + "missed_rescue": 146, + "avoided_harm": 63, + "selected_both_correct": 3670, + "selected_both_wrong": 277 + }, + { + "budget": 0.8, + "threshold": -3.4036288261413574, + "operator": "ge", + "actual_fraction": 0.7999140647378975, + "metric": 0.8877112575193354, + "rescued": 1250, + "harmed": 328, + "missed_rescue": 139, + "avoided_harm": 60, + "selected_both_correct": 3730, + "selected_both_wrong": 277 + }, + { + "budget": 0.81, + "threshold": -3.4344234466552734, + "operator": "ge", + "actual_fraction": 0.8099398453165282, + "metric": 0.8877112575193354, + "rescued": 1256, + "harmed": 334, + "missed_rescue": 133, + "avoided_harm": 54, + "selected_both_correct": 3783, + "selected_both_wrong": 282 + }, + { + "budget": 0.82, + "threshold": -3.4733219146728516, + "operator": "ge", + "actual_fraction": 0.819965625895159, + "metric": 0.8882841592666858, + "rescued": 1264, + "harmed": 338, + "missed_rescue": 125, + "avoided_harm": 50, + "selected_both_correct": 3841, + "selected_both_wrong": 282 + }, + { + "budget": 0.83, + "threshold": -3.513460636138916, + "operator": "ge", + "actual_fraction": 0.8299914064737898, + "metric": 0.888857061014036, + "rescued": 1271, + "harmed": 341, + "missed_rescue": 118, + "avoided_harm": 47, + "selected_both_correct": 3898, + "selected_both_wrong": 285 + }, + { + "budget": 0.84, + "threshold": -3.55360746383667, + "operator": "ge", + "actual_fraction": 0.8398739616155829, + "metric": 0.8894299627613864, + "rescued": 1277, + "harmed": 343, + "missed_rescue": 112, + "avoided_harm": 45, + "selected_both_correct": 3957, + "selected_both_wrong": 287 + }, + { + "budget": 0.85, + "threshold": -3.588076114654541, + "operator": "ge", + "actual_fraction": 0.8498997421942137, + "metric": 0.8904325408192495, + "rescued": 1285, + "harmed": 344, + "missed_rescue": 104, + "avoided_harm": 44, + "selected_both_correct": 4018, + "selected_both_wrong": 287 + }, + { + "budget": 0.86, + "threshold": -3.627887725830078, + "operator": "ge", + "actual_fraction": 0.8599255227728445, + "metric": 0.8911486680034374, + "rescued": 1292, + "harmed": 346, + "missed_rescue": 97, + "avoided_harm": 42, + "selected_both_correct": 4077, + "selected_both_wrong": 289 + }, + { + "budget": 0.87, + "threshold": -3.6658315658569336, + "operator": "ge", + "actual_fraction": 0.8699513033514752, + "metric": 0.8918647951876253, + "rescued": 1299, + "harmed": 348, + "missed_rescue": 90, + "avoided_harm": 40, + "selected_both_correct": 4137, + "selected_both_wrong": 290 + }, + { + "budget": 0.88, + "threshold": -3.7103419303894043, + "operator": "ge", + "actual_fraction": 0.879977083930106, + "metric": 0.8931538241191636, + "rescued": 1311, + "harmed": 351, + "missed_rescue": 78, + "avoided_harm": 37, + "selected_both_correct": 4191, + "selected_both_wrong": 291 + }, + { + "budget": 0.89, + "threshold": -3.765810012817383, + "operator": "ge", + "actual_fraction": 0.8898596390718991, + "metric": 0.893010598682326, + "rescued": 1316, + "harmed": 357, + "missed_rescue": 73, + "avoided_harm": 31, + "selected_both_correct": 4247, + "selected_both_wrong": 293 + }, + { + "budget": 0.9, + "threshold": -3.8223915100097656, + "operator": "ge", + "actual_fraction": 0.8998854196505299, + "metric": 0.893010598682326, + "rescued": 1320, + "harmed": 361, + "missed_rescue": 69, + "avoided_harm": 27, + "selected_both_correct": 4303, + "selected_both_wrong": 299 + }, + { + "budget": 0.91, + "threshold": -3.864851236343384, + "operator": "ge", + "actual_fraction": 0.9099112002291607, + "metric": 0.8938699513033515, + "rescued": 1328, + "harmed": 363, + "missed_rescue": 61, + "avoided_harm": 25, + "selected_both_correct": 4362, + "selected_both_wrong": 300 + }, + { + "budget": 0.92, + "threshold": -3.9261999130249023, + "operator": "ge", + "actual_fraction": 0.9199369808077915, + "metric": 0.8942996276138642, + "rescued": 1334, + "harmed": 366, + "missed_rescue": 55, + "avoided_harm": 22, + "selected_both_correct": 4421, + "selected_both_wrong": 302 + }, + { + "budget": 0.93, + "threshold": -3.9741241931915283, + "operator": "ge", + "actual_fraction": 0.9299627613864222, + "metric": 0.8951589802348897, + "rescued": 1342, + "harmed": 368, + "missed_rescue": 47, + "avoided_harm": 20, + "selected_both_correct": 4481, + "selected_both_wrong": 302 + }, + { + "budget": 0.94, + "threshold": -4.04119873046875, + "operator": "ge", + "actual_fraction": 0.939988541965053, + "metric": 0.8955886565454024, + "rescued": 1347, + "harmed": 370, + "missed_rescue": 42, + "avoided_harm": 18, + "selected_both_correct": 4541, + "selected_both_wrong": 305 + }, + { + "budget": 0.95, + "threshold": -4.1099114418029785, + "operator": "ge", + "actual_fraction": 0.9498710971068461, + "metric": 0.8965912346032655, + "rescued": 1356, + "harmed": 372, + "missed_rescue": 33, + "avoided_harm": 16, + "selected_both_correct": 4596, + "selected_both_wrong": 308 + }, + { + "budget": 0.96, + "threshold": -4.188605308532715, + "operator": "ge", + "actual_fraction": 0.9598968776854769, + "metric": 0.8963047837295903, + "rescued": 1360, + "harmed": 378, + "missed_rescue": 29, + "avoided_harm": 10, + "selected_both_correct": 4656, + "selected_both_wrong": 308 + }, + { + "budget": 0.97, + "threshold": -4.283358573913574, + "operator": "ge", + "actual_fraction": 0.9699226582641077, + "metric": 0.896448009166428, + "rescued": 1366, + "harmed": 383, + "missed_rescue": 23, + "avoided_harm": 5, + "selected_both_correct": 4715, + "selected_both_wrong": 308 + }, + { + "budget": 0.98, + "threshold": -4.372988224029541, + "operator": "ge", + "actual_fraction": 0.9799484388427385, + "metric": 0.8971641363506159, + "rescued": 1372, + "harmed": 384, + "missed_rescue": 17, + "avoided_harm": 4, + "selected_both_correct": 4776, + "selected_both_wrong": 310 + }, + { + "budget": 0.99, + "threshold": -4.5110697746276855, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.898166714408479, + "rescued": 1381, + "harmed": 386, + "missed_rescue": 8, + "avoided_harm": 2, + "selected_both_correct": 4833, + "selected_both_wrong": 312 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8990260670295045, + "rescued": 1389, + "harmed": 388, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 4888, + "selected_both_wrong": 317 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7642247255506924, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2706, + "avoided_harm": 711, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.4486454725265503, + "operator": "ge", + "actual_fraction": 0.014135036234483749, + "metric": 0.7686733156346416, + "rescued": 70, + "harmed": 8, + "missed_rescue": 2636, + "avoided_harm": 703, + "selected_both_correct": 110, + "selected_both_wrong": 9 + }, + { + "budget": 0.02, + "threshold": 1.3545215129852295, + "operator": "ge", + "actual_fraction": 0.022099447513812154, + "metric": 0.7713998708473847, + "rescued": 112, + "harmed": 12, + "missed_rescue": 2594, + "avoided_harm": 699, + "selected_both_correct": 168, + "selected_both_wrong": 16 + }, + { + "budget": 0.03, + "threshold": 1.2635189294815063, + "operator": "ge", + "actual_fraction": 0.03185764511731363, + "metric": 0.7752744493075985, + "rescued": 170, + "harmed": 16, + "missed_rescue": 2536, + "avoided_harm": 695, + "selected_both_correct": 233, + "selected_both_wrong": 25 + }, + { + "budget": 0.04, + "threshold": 1.178093433380127, + "operator": "ge", + "actual_fraction": 0.04075482528521202, + "metric": 0.7786467675970439, + "rescued": 222, + "harmed": 21, + "missed_rescue": 2484, + "avoided_harm": 690, + "selected_both_correct": 292, + "selected_both_wrong": 33 + }, + { + "budget": 0.05, + "threshold": 1.101384162902832, + "operator": "ge", + "actual_fraction": 0.05166104613618426, + "metric": 0.7823778431513239, + "rescued": 281, + "harmed": 28, + "missed_rescue": 2425, + "avoided_harm": 683, + "selected_both_correct": 371, + "selected_both_wrong": 40 + }, + { + "budget": 0.06, + "threshold": 1.0296903848648071, + "operator": "ge", + "actual_fraction": 0.06177800100452034, + "metric": 0.7846738896462654, + "rescued": 324, + "harmed": 39, + "missed_rescue": 2382, + "avoided_harm": 672, + "selected_both_correct": 451, + "selected_both_wrong": 47 + }, + { + "budget": 0.07, + "threshold": 0.9584200978279114, + "operator": "ge", + "actual_fraction": 0.07289947621439334, + "metric": 0.7876156992179092, + "rescued": 377, + "harmed": 51, + "missed_rescue": 2329, + "avoided_harm": 660, + "selected_both_correct": 532, + "selected_both_wrong": 56 + }, + { + "budget": 0.08, + "threshold": 0.9015151262283325, + "operator": "ge", + "actual_fraction": 0.08150965057042404, + "metric": 0.7899834971658176, + "rescued": 420, + "harmed": 61, + "missed_rescue": 2286, + "avoided_harm": 650, + "selected_both_correct": 587, + "selected_both_wrong": 68 + }, + { + "budget": 0.09, + "threshold": 0.8391129970550537, + "operator": "ge", + "actual_fraction": 0.09184185979766091, + "metric": 0.7930688096433953, + "rescued": 476, + "harmed": 74, + "missed_rescue": 2230, + "avoided_harm": 637, + "selected_both_correct": 652, + "selected_both_wrong": 78 + }, + { + "budget": 0.1, + "threshold": 0.7945014238357544, + "operator": "ge", + "actual_fraction": 0.09865824782951854, + "metric": 0.7949343474205353, + "rescued": 511, + "harmed": 83, + "missed_rescue": 2195, + "avoided_harm": 628, + "selected_both_correct": 699, + "selected_both_wrong": 82 + }, + { + "budget": 0.11, + "threshold": 0.7454464435577393, + "operator": "ge", + "actual_fraction": 0.10619215039104542, + "metric": 0.7968716366506422, + "rescued": 549, + "harmed": 94, + "missed_rescue": 2157, + "avoided_harm": 617, + "selected_both_correct": 743, + "selected_both_wrong": 94 + }, + { + "budget": 0.12, + "threshold": 0.7009376883506775, + "operator": "ge", + "actual_fraction": 0.1150175791059769, + "metric": 0.8003157063930545, + "rescued": 602, + "harmed": 99, + "missed_rescue": 2104, + "avoided_harm": 612, + "selected_both_correct": 794, + "selected_both_wrong": 108 + }, + { + "budget": 0.13, + "threshold": 0.651968240737915, + "operator": "ge", + "actual_fraction": 0.12326899619717299, + "metric": 0.8028270072468967, + "rescued": 645, + "harmed": 107, + "missed_rescue": 2061, + "avoided_harm": 604, + "selected_both_correct": 848, + "selected_both_wrong": 118 + }, + { + "budget": 0.14, + "threshold": 0.5855442881584167, + "operator": "ge", + "actual_fraction": 0.13424696850111215, + "metric": 0.8055535624596398, + "rescued": 697, + "harmed": 121, + "missed_rescue": 2009, + "avoided_harm": 590, + "selected_both_correct": 923, + "selected_both_wrong": 130 + }, + { + "budget": 0.15, + "threshold": 0.5396223068237305, + "operator": "ge", + "actual_fraction": 0.14192437396857285, + "metric": 0.8077778575016145, + "rescued": 732, + "harmed": 125, + "missed_rescue": 1974, + "avoided_harm": 586, + "selected_both_correct": 981, + "selected_both_wrong": 140 + }, + { + "budget": 0.16, + "threshold": 0.4906216263771057, + "operator": "ge", + "actual_fraction": 0.15003228815383512, + "metric": 0.8095716438257875, + "rescued": 764, + "harmed": 132, + "missed_rescue": 1942, + "avoided_harm": 579, + "selected_both_correct": 1047, + "selected_both_wrong": 148 + }, + { + "budget": 0.17, + "threshold": 0.44199177622795105, + "operator": "ge", + "actual_fraction": 0.15900121977470044, + "metric": 0.8120111932266628, + "rescued": 807, + "harmed": 141, + "missed_rescue": 1899, + "avoided_harm": 570, + "selected_both_correct": 1113, + "selected_both_wrong": 155 + }, + { + "budget": 0.18, + "threshold": 0.3899296522140503, + "operator": "ge", + "actual_fraction": 0.16725263686589653, + "metric": 0.813733228097869, + "rescued": 844, + "harmed": 154, + "missed_rescue": 1862, + "avoided_harm": 557, + "selected_both_correct": 1171, + "selected_both_wrong": 162 + }, + { + "budget": 0.19, + "threshold": 0.325008362531662, + "operator": "ge", + "actual_fraction": 0.1776565975461003, + "metric": 0.8158857716868767, + "rescued": 887, + "harmed": 167, + "missed_rescue": 1819, + "avoided_harm": 544, + "selected_both_correct": 1246, + "selected_both_wrong": 176 + }, + { + "budget": 0.2, + "threshold": 0.26310965418815613, + "operator": "ge", + "actual_fraction": 0.18806055822630408, + "metric": 0.8191145870703882, + "rescued": 946, + "harmed": 181, + "missed_rescue": 1760, + "avoided_harm": 530, + "selected_both_correct": 1308, + "selected_both_wrong": 186 + }, + { + "budget": 0.21, + "threshold": 0.19935977458953857, + "operator": "ge", + "actual_fraction": 0.1980340101887063, + "metric": 0.821123627753462, + "rescued": 987, + "harmed": 194, + "missed_rescue": 1719, + "avoided_harm": 517, + "selected_both_correct": 1381, + "selected_both_wrong": 198 + }, + { + "budget": 0.22, + "threshold": 0.13677698373794556, + "operator": "ge", + "actual_fraction": 0.20836621941594316, + "metric": 0.823850182966205, + "rescued": 1038, + "harmed": 207, + "missed_rescue": 1668, + "avoided_harm": 504, + "selected_both_correct": 1449, + "selected_both_wrong": 210 + }, + { + "budget": 0.23, + "threshold": 0.0732785016298294, + "operator": "ge", + "actual_fraction": 0.2170481452249408, + "metric": 0.8260027265552128, + "rescued": 1082, + "harmed": 221, + "missed_rescue": 1624, + "avoided_harm": 490, + "selected_both_correct": 1505, + "selected_both_wrong": 217 + }, + { + "budget": 0.24, + "threshold": 0.008367598056793213, + "operator": "ge", + "actual_fraction": 0.22860012915261535, + "metric": 0.828514027409055, + "rescued": 1130, + "harmed": 234, + "missed_rescue": 1576, + "avoided_harm": 477, + "selected_both_correct": 1597, + "selected_both_wrong": 225 + }, + { + "budget": 0.25, + "threshold": -0.0719703733921051, + "operator": "ge", + "actual_fraction": 0.2414436392336945, + "metric": 0.8324603573222358, + "rescued": 1194, + "harmed": 243, + "missed_rescue": 1512, + "avoided_harm": 468, + "selected_both_correct": 1693, + "selected_both_wrong": 235 + }, + { + "budget": 0.26, + "threshold": -0.14035016298294067, + "operator": "ge", + "actual_fraction": 0.2526368658965344, + "metric": 0.834110640740475, + "rescued": 1229, + "harmed": 255, + "missed_rescue": 1477, + "avoided_harm": 456, + "selected_both_correct": 1789, + "selected_both_wrong": 248 + }, + { + "budget": 0.27, + "threshold": -0.2074521780014038, + "operator": "ge", + "actual_fraction": 0.2628973236708043, + "metric": 0.8366219415943172, + "rescued": 1273, + "harmed": 264, + "missed_rescue": 1433, + "avoided_harm": 447, + "selected_both_correct": 1874, + "selected_both_wrong": 253 + }, + { + "budget": 0.28, + "threshold": -0.274451345205307, + "operator": "ge", + "actual_fraction": 0.27545382794001577, + "metric": 0.839635502618928, + "rescued": 1324, + "harmed": 273, + "missed_rescue": 1382, + "avoided_harm": 438, + "selected_both_correct": 1977, + "selected_both_wrong": 265 + }, + { + "budget": 0.29, + "threshold": -0.331489622592926, + "operator": "ge", + "actual_fraction": 0.28298773050154263, + "metric": 0.8412140345842003, + "rescued": 1354, + "harmed": 281, + "missed_rescue": 1352, + "avoided_harm": 430, + "selected_both_correct": 2038, + "selected_both_wrong": 271 + }, + { + "budget": 0.3, + "threshold": -0.4019598960876465, + "operator": "ge", + "actual_fraction": 0.2949702231470187, + "metric": 0.8435818325321087, + "rescued": 1403, + "harmed": 297, + "missed_rescue": 1303, + "avoided_harm": 414, + "selected_both_correct": 2134, + "selected_both_wrong": 277 + }, + { + "budget": 0.31, + "threshold": -0.4725560247898102, + "operator": "ge", + "actual_fraction": 0.3060916983568917, + "metric": 0.8468106479156203, + "rescued": 1458, + "harmed": 307, + "missed_rescue": 1248, + "avoided_harm": 404, + "selected_both_correct": 2214, + "selected_both_wrong": 287 + }, + { + "budget": 0.32, + "threshold": -0.5430232882499695, + "operator": "ge", + "actual_fraction": 0.31886345698500396, + "metric": 0.8496807060342972, + "rescued": 1507, + "harmed": 316, + "missed_rescue": 1199, + "avoided_harm": 395, + "selected_both_correct": 2323, + "selected_both_wrong": 298 + }, + { + "budget": 0.33, + "threshold": -0.6012662649154663, + "operator": "ge", + "actual_fraction": 0.32740187988806774, + "metric": 0.8509722321877018, + "rescued": 1537, + "harmed": 328, + "missed_rescue": 1169, + "avoided_harm": 383, + "selected_both_correct": 2394, + "selected_both_wrong": 304 + }, + { + "budget": 0.34, + "threshold": -0.6759253740310669, + "operator": "ge", + "actual_fraction": 0.34117815885771685, + "metric": 0.8543445504771472, + "rescued": 1591, + "harmed": 335, + "missed_rescue": 1115, + "avoided_harm": 376, + "selected_both_correct": 2516, + "selected_both_wrong": 313 + }, + { + "budget": 0.35, + "threshold": -0.7336556911468506, + "operator": "ge", + "actual_fraction": 0.350290593384516, + "metric": 0.8566405969720887, + "rescued": 1625, + "harmed": 337, + "missed_rescue": 1081, + "avoided_harm": 374, + "selected_both_correct": 2600, + "selected_both_wrong": 320 + }, + { + "budget": 0.36, + "threshold": -0.7973066568374634, + "operator": "ge", + "actual_fraction": 0.3600487909880175, + "metric": 0.8589366434670302, + "rescued": 1662, + "harmed": 342, + "missed_rescue": 1044, + "avoided_harm": 369, + "selected_both_correct": 2689, + "selected_both_wrong": 325 + }, + { + "budget": 0.37, + "threshold": -0.86806321144104, + "operator": "ge", + "actual_fraction": 0.3700939944033867, + "metric": 0.8609456841501041, + "rescued": 1696, + "harmed": 348, + "missed_rescue": 1010, + "avoided_harm": 363, + "selected_both_correct": 2782, + "selected_both_wrong": 332 + }, + { + "budget": 0.38, + "threshold": -0.9553278684616089, + "operator": "ge", + "actual_fraction": 0.3815742268780943, + "metric": 0.8636004879098802, + "rescued": 1741, + "harmed": 356, + "missed_rescue": 965, + "avoided_harm": 355, + "selected_both_correct": 2882, + "selected_both_wrong": 339 + }, + { + "budget": 0.39, + "threshold": -1.0301940441131592, + "operator": "ge", + "actual_fraction": 0.39039965559302575, + "metric": 0.8661835402166894, + "rescued": 1784, + "harmed": 363, + "missed_rescue": 922, + "avoided_harm": 348, + "selected_both_correct": 2949, + "selected_both_wrong": 345 + }, + { + "budget": 0.4, + "threshold": -1.1348119974136353, + "operator": "ge", + "actual_fraction": 0.40302791131520416, + "metric": 0.8676185692760279, + "rescued": 1817, + "harmed": 376, + "missed_rescue": 889, + "avoided_harm": 335, + "selected_both_correct": 3075, + "selected_both_wrong": 349 + }, + { + "budget": 0.41, + "threshold": -1.226643681526184, + "operator": "ge", + "actual_fraction": 0.4119968429360695, + "metric": 0.8688383439764655, + "rescued": 1843, + "harmed": 385, + "missed_rescue": 863, + "avoided_harm": 326, + "selected_both_correct": 3153, + "selected_both_wrong": 361 + }, + { + "budget": 0.42, + "threshold": -1.2974129915237427, + "operator": "ge", + "actual_fraction": 0.4199612542153979, + "metric": 0.8699863672239363, + "rescued": 1865, + "harmed": 391, + "missed_rescue": 841, + "avoided_harm": 320, + "selected_both_correct": 3231, + "selected_both_wrong": 366 + }, + { + "budget": 0.43, + "threshold": -1.4153810739517212, + "operator": "ge", + "actual_fraction": 0.4302217119896678, + "metric": 0.8722106622659109, + "rescued": 1900, + "harmed": 395, + "missed_rescue": 806, + "avoided_harm": 316, + "selected_both_correct": 3326, + "selected_both_wrong": 375 + }, + { + "budget": 0.44, + "threshold": -1.5353657007217407, + "operator": "ge", + "actual_fraction": 0.44098442993470616, + "metric": 0.8736456913252494, + "rescued": 1928, + "harmed": 403, + "missed_rescue": 778, + "avoided_harm": 308, + "selected_both_correct": 3436, + "selected_both_wrong": 379 + }, + { + "budget": 0.45, + "threshold": -1.6688950061798096, + "operator": "ge", + "actual_fraction": 0.4515318935208438, + "metric": 0.8753677261964555, + "rescued": 1961, + "harmed": 412, + "missed_rescue": 745, + "avoided_harm": 299, + "selected_both_correct": 3537, + "selected_both_wrong": 383 + }, + { + "budget": 0.46, + "threshold": -1.7557787895202637, + "operator": "ge", + "actual_fraction": 0.460644328047643, + "metric": 0.8768027552557939, + "rescued": 1984, + "harmed": 415, + "missed_rescue": 722, + "avoided_harm": 296, + "selected_both_correct": 3633, + "selected_both_wrong": 388 + }, + { + "budget": 0.47, + "threshold": -1.841616153717041, + "operator": "ge", + "actual_fraction": 0.47040252565114443, + "metric": 0.877735524144364, + "rescued": 2004, + "harmed": 422, + "missed_rescue": 702, + "avoided_harm": 289, + "selected_both_correct": 3736, + "selected_both_wrong": 394 + }, + { + "budget": 0.48, + "threshold": -1.9284571409225464, + "operator": "ge", + "actual_fraction": 0.47929970581904285, + "metric": 0.878596541579967, + "rescued": 2023, + "harmed": 429, + "missed_rescue": 683, + "avoided_harm": 282, + "selected_both_correct": 3832, + "selected_both_wrong": 396 + }, + { + "budget": 0.49, + "threshold": -2.0005507469177246, + "operator": "ge", + "actual_fraction": 0.48912965487551124, + "metric": 0.8793858075626031, + "rescued": 2042, + "harmed": 437, + "missed_rescue": 664, + "avoided_harm": 274, + "selected_both_correct": 3938, + "selected_both_wrong": 400 + }, + { + "budget": 0.5, + "threshold": -2.0627641677856445, + "operator": "ge", + "actual_fraction": 0.497668077778575, + "metric": 0.8803185764511732, + "rescued": 2059, + "harmed": 441, + "missed_rescue": 647, + "avoided_harm": 270, + "selected_both_correct": 4032, + "selected_both_wrong": 404 + }, + { + "budget": 0.51, + "threshold": -2.1383652687072754, + "operator": "ge", + "actual_fraction": 0.508789552988448, + "metric": 0.8811795938867762, + "rescued": 2080, + "harmed": 450, + "missed_rescue": 626, + "avoided_harm": 261, + "selected_both_correct": 4153, + "selected_both_wrong": 408 + }, + { + "budget": 0.52, + "threshold": -2.2056591510772705, + "operator": "ge", + "actual_fraction": 0.5194087680275525, + "metric": 0.8819688598694123, + "rescued": 2094, + "harmed": 453, + "missed_rescue": 612, + "avoided_harm": 258, + "selected_both_correct": 4278, + "selected_both_wrong": 414 + }, + { + "budget": 0.53, + "threshold": -2.2718443870544434, + "operator": "ge", + "actual_fraction": 0.5305302432374256, + "metric": 0.8828298773050154, + "rescued": 2113, + "harmed": 460, + "missed_rescue": 593, + "avoided_harm": 251, + "selected_both_correct": 4406, + "selected_both_wrong": 415 + }, + { + "budget": 0.54, + "threshold": -2.325044631958008, + "operator": "ge", + "actual_fraction": 0.5404319437468609, + "metric": 0.8835473918346847, + "rescued": 2126, + "harmed": 463, + "missed_rescue": 580, + "avoided_harm": 248, + "selected_both_correct": 4524, + "selected_both_wrong": 419 + }, + { + "budget": 0.55, + "threshold": -2.380568504333496, + "operator": "ge", + "actual_fraction": 0.5485398579321231, + "metric": 0.8844801607232546, + "rescued": 2141, + "harmed": 465, + "missed_rescue": 565, + "avoided_harm": 246, + "selected_both_correct": 4618, + "selected_both_wrong": 421 + }, + { + "budget": 0.56, + "threshold": -2.431140184402466, + "operator": "ge", + "actual_fraction": 0.5583698069885915, + "metric": 0.88505417234699, + "rescued": 2156, + "harmed": 472, + "missed_rescue": 550, + "avoided_harm": 239, + "selected_both_correct": 4732, + "selected_both_wrong": 422 + }, + { + "budget": 0.57, + "threshold": -2.4825475215911865, + "operator": "ge", + "actual_fraction": 0.568128004592093, + "metric": 0.8851976752529239, + "rescued": 2164, + "harmed": 478, + "missed_rescue": 542, + "avoided_harm": 233, + "selected_both_correct": 4850, + "selected_both_wrong": 426 + }, + { + "budget": 0.58, + "threshold": -2.532240867614746, + "operator": "ge", + "actual_fraction": 0.5800387457846021, + "metric": 0.8856281839707254, + "rescued": 2177, + "harmed": 485, + "missed_rescue": 529, + "avoided_harm": 226, + "selected_both_correct": 4991, + "selected_both_wrong": 431 + }, + { + "budget": 0.59, + "threshold": -2.575655460357666, + "operator": "ge", + "actual_fraction": 0.5884336657817321, + "metric": 0.8861304441414939, + "rescued": 2193, + "harmed": 494, + "missed_rescue": 513, + "avoided_harm": 217, + "selected_both_correct": 5079, + "selected_both_wrong": 435 + }, + { + "budget": 0.6, + "threshold": -2.627545118331909, + "operator": "ge", + "actual_fraction": 0.5996986438975389, + "metric": 0.8867762072181962, + "rescued": 2206, + "harmed": 498, + "missed_rescue": 500, + "avoided_harm": 213, + "selected_both_correct": 5212, + "selected_both_wrong": 442 + }, + { + "budget": 0.61, + "threshold": -2.667684316635132, + "operator": "ge", + "actual_fraction": 0.6089545813302719, + "metric": 0.8871349644830308, + "rescued": 2217, + "harmed": 504, + "missed_rescue": 489, + "avoided_harm": 207, + "selected_both_correct": 5322, + "selected_both_wrong": 444 + }, + { + "budget": 0.62, + "threshold": -2.7069380283355713, + "operator": "ge", + "actual_fraction": 0.6171342469685012, + "metric": 0.887780727559733, + "rescued": 2227, + "harmed": 505, + "missed_rescue": 479, + "avoided_harm": 206, + "selected_both_correct": 5422, + "selected_both_wrong": 447 + }, + { + "budget": 0.63, + "threshold": -2.7543530464172363, + "operator": "ge", + "actual_fraction": 0.6279687163665064, + "metric": 0.8882829877305015, + "rescued": 2237, + "harmed": 508, + "missed_rescue": 469, + "avoided_harm": 203, + "selected_both_correct": 5553, + "selected_both_wrong": 454 + }, + { + "budget": 0.64, + "threshold": -2.797299861907959, + "operator": "ge", + "actual_fraction": 0.6380856712348425, + "metric": 0.8887134964483031, + "rescued": 2248, + "harmed": 513, + "missed_rescue": 458, + "avoided_harm": 198, + "selected_both_correct": 5677, + "selected_both_wrong": 455 + }, + { + "budget": 0.65, + "threshold": -2.84092378616333, + "operator": "ge", + "actual_fraction": 0.6494941522565832, + "metric": 0.8899332711487408, + "rescued": 2268, + "harmed": 516, + "missed_rescue": 438, + "avoided_harm": 195, + "selected_both_correct": 5810, + "selected_both_wrong": 458 + }, + { + "budget": 0.66, + "threshold": -2.878750801086426, + "operator": "ge", + "actual_fraction": 0.659037095501184, + "metric": 0.8900050226017077, + "rescued": 2279, + "harmed": 526, + "missed_rescue": 427, + "avoided_harm": 185, + "selected_both_correct": 5921, + "selected_both_wrong": 459 + }, + { + "budget": 0.67, + "threshold": -2.9207773208618164, + "operator": "ge", + "actual_fraction": 0.6699433163521561, + "metric": 0.890722537131377, + "rescued": 2296, + "harmed": 533, + "missed_rescue": 410, + "avoided_harm": 178, + "selected_both_correct": 6045, + "selected_both_wrong": 463 + }, + { + "budget": 0.68, + "threshold": -2.963459014892578, + "operator": "ge", + "actual_fraction": 0.6799167683145584, + "metric": 0.8914400516610461, + "rescued": 2309, + "harmed": 536, + "missed_rescue": 397, + "avoided_harm": 175, + "selected_both_correct": 6163, + "selected_both_wrong": 468 + }, + { + "budget": 0.69, + "threshold": -3.001129627227783, + "operator": "ge", + "actual_fraction": 0.6891727057472914, + "metric": 0.8917270574729138, + "rescued": 2318, + "harmed": 541, + "missed_rescue": 388, + "avoided_harm": 170, + "selected_both_correct": 6275, + "selected_both_wrong": 471 + }, + { + "budget": 0.7, + "threshold": -3.040706157684326, + "operator": "ge", + "actual_fraction": 0.7009399440338667, + "metric": 0.8921575661907154, + "rescued": 2328, + "harmed": 545, + "missed_rescue": 378, + "avoided_harm": 166, + "selected_both_correct": 6420, + "selected_both_wrong": 476 + }, + { + "budget": 0.71, + "threshold": -3.0831708908081055, + "operator": "ge", + "actual_fraction": 0.7133529453971443, + "metric": 0.8929468321733515, + "rescued": 2345, + "harmed": 551, + "missed_rescue": 361, + "avoided_harm": 160, + "selected_both_correct": 6565, + "selected_both_wrong": 481 + }, + { + "budget": 0.72, + "threshold": -3.121913433074951, + "operator": "ge", + "actual_fraction": 0.7240439118892158, + "metric": 0.8938796010619215, + "rescued": 2361, + "harmed": 554, + "missed_rescue": 345, + "avoided_harm": 157, + "selected_both_correct": 6693, + "selected_both_wrong": 483 + }, + { + "budget": 0.73, + "threshold": -3.1643247604370117, + "operator": "ge", + "actual_fraction": 0.7343043696634857, + "metric": 0.8945971155915907, + "rescued": 2375, + "harmed": 558, + "missed_rescue": 331, + "avoided_harm": 153, + "selected_both_correct": 6817, + "selected_both_wrong": 484 + }, + { + "budget": 0.74, + "threshold": -3.1954593658447266, + "operator": "ge", + "actual_fraction": 0.7427710411135825, + "metric": 0.8954581330271938, + "rescued": 2390, + "harmed": 561, + "missed_rescue": 316, + "avoided_harm": 150, + "selected_both_correct": 6916, + "selected_both_wrong": 485 + }, + { + "budget": 0.75, + "threshold": -3.230159282684326, + "operator": "ge", + "actual_fraction": 0.7531032503408194, + "metric": 0.8961756475568631, + "rescued": 2403, + "harmed": 564, + "missed_rescue": 303, + "avoided_harm": 147, + "selected_both_correct": 7042, + "selected_both_wrong": 487 + }, + { + "budget": 0.76, + "threshold": -3.2606701850891113, + "operator": "ge", + "actual_fraction": 0.7622156848676186, + "metric": 0.8963909019157638, + "rescued": 2412, + "harmed": 570, + "missed_rescue": 294, + "avoided_harm": 141, + "selected_both_correct": 7148, + "selected_both_wrong": 493 + }, + { + "budget": 0.77, + "threshold": -3.3036723136901855, + "operator": "ge", + "actual_fraction": 0.7745569347779293, + "metric": 0.8975389251632345, + "rescued": 2434, + "harmed": 576, + "missed_rescue": 272, + "avoided_harm": 135, + "selected_both_correct": 7287, + "selected_both_wrong": 498 + }, + { + "budget": 0.78, + "threshold": -3.3321590423583984, + "operator": "ge", + "actual_fraction": 0.7818755829805554, + "metric": 0.8977541795221353, + "rescued": 2441, + "harmed": 580, + "missed_rescue": 265, + "avoided_harm": 131, + "selected_both_correct": 7374, + "selected_both_wrong": 502 + }, + { + "budget": 0.79, + "threshold": -3.368250846862793, + "operator": "ge", + "actual_fraction": 0.7897682428069168, + "metric": 0.8981129367869699, + "rescued": 2451, + "harmed": 585, + "missed_rescue": 255, + "avoided_harm": 126, + "selected_both_correct": 7467, + "selected_both_wrong": 504 + }, + { + "budget": 0.8, + "threshold": -3.4036288261413574, + "operator": "ge", + "actual_fraction": 0.798880677333716, + "metric": 0.8978976824280692, + "rescued": 2456, + "harmed": 593, + "missed_rescue": 250, + "avoided_harm": 118, + "selected_both_correct": 7579, + "selected_both_wrong": 506 + }, + { + "budget": 0.81, + "threshold": -3.4344234466552734, + "operator": "ge", + "actual_fraction": 0.8069885915189783, + "metric": 0.8983281911458707, + "rescued": 2465, + "harmed": 596, + "missed_rescue": 241, + "avoided_harm": 115, + "selected_both_correct": 7679, + "selected_both_wrong": 507 + }, + { + "budget": 0.82, + "threshold": -3.4733219146728516, + "operator": "ge", + "actual_fraction": 0.8178230609169835, + "metric": 0.8991174571285069, + "rescued": 2480, + "harmed": 600, + "missed_rescue": 226, + "avoided_harm": 111, + "selected_both_correct": 7806, + "selected_both_wrong": 512 + }, + { + "budget": 0.83, + "threshold": -3.513460636138916, + "operator": "ge", + "actual_fraction": 0.8289445361268566, + "metric": 0.8997632202052092, + "rescued": 2500, + "harmed": 611, + "missed_rescue": 206, + "avoided_harm": 100, + "selected_both_correct": 7927, + "selected_both_wrong": 515 + }, + { + "budget": 0.84, + "threshold": -3.55360746383667, + "operator": "ge", + "actual_fraction": 0.8394919997129942, + "metric": 0.9006242376408122, + "rescued": 2513, + "harmed": 612, + "missed_rescue": 193, + "avoided_harm": 99, + "selected_both_correct": 8059, + "selected_both_wrong": 516 + }, + { + "budget": 0.85, + "threshold": -3.588076114654541, + "operator": "ge", + "actual_fraction": 0.8486044342397934, + "metric": 0.9005524861878453, + "rescued": 2522, + "harmed": 622, + "missed_rescue": 184, + "avoided_harm": 89, + "selected_both_correct": 8162, + "selected_both_wrong": 521 + }, + { + "budget": 0.86, + "threshold": -3.627887725830078, + "operator": "ge", + "actual_fraction": 0.8575733658606587, + "metric": 0.9006242376408122, + "rescued": 2531, + "harmed": 630, + "missed_rescue": 175, + "avoided_harm": 81, + "selected_both_correct": 8266, + "selected_both_wrong": 525 + }, + { + "budget": 0.87, + "threshold": -3.6658315658569336, + "operator": "ge", + "actual_fraction": 0.8658247829518547, + "metric": 0.9005524861878453, + "rescued": 2538, + "harmed": 638, + "missed_rescue": 168, + "avoided_harm": 73, + "selected_both_correct": 8363, + "selected_both_wrong": 528 + }, + { + "budget": 0.88, + "threshold": -3.7103419303894043, + "operator": "ge", + "actual_fraction": 0.8770897610676616, + "metric": 0.9006959890937791, + "rescued": 2546, + "harmed": 644, + "missed_rescue": 160, + "avoided_harm": 67, + "selected_both_correct": 8501, + "selected_both_wrong": 533 + }, + { + "budget": 0.89, + "threshold": -3.765810012817383, + "operator": "ge", + "actual_fraction": 0.8898615196957739, + "metric": 0.9017005094353161, + "rescued": 2564, + "harmed": 648, + "missed_rescue": 142, + "avoided_harm": 63, + "selected_both_correct": 8651, + "selected_both_wrong": 539 + }, + { + "budget": 0.9, + "threshold": -3.8223915100097656, + "operator": "ge", + "actual_fraction": 0.9030637870416875, + "metric": 0.9029920355887207, + "rescued": 2589, + "harmed": 655, + "missed_rescue": 117, + "avoided_harm": 56, + "selected_both_correct": 8800, + "selected_both_wrong": 542 + }, + { + "budget": 0.91, + "threshold": -3.864851236343384, + "operator": "ge", + "actual_fraction": 0.9118174643036522, + "metric": 0.9032790414005883, + "rescued": 2598, + "harmed": 660, + "missed_rescue": 108, + "avoided_harm": 51, + "selected_both_correct": 8906, + "selected_both_wrong": 544 + }, + { + "budget": 0.92, + "threshold": -3.9261999130249023, + "operator": "ge", + "actual_fraction": 0.9236564540431944, + "metric": 0.9041400588361914, + "rescued": 2611, + "harmed": 661, + "missed_rescue": 95, + "avoided_harm": 50, + "selected_both_correct": 9051, + "selected_both_wrong": 550 + }, + { + "budget": 0.93, + "threshold": -3.9741241931915283, + "operator": "ge", + "actual_fraction": 0.9316926167754898, + "metric": 0.904570567553993, + "rescued": 2623, + "harmed": 667, + "missed_rescue": 83, + "avoided_harm": 44, + "selected_both_correct": 9140, + "selected_both_wrong": 555 + }, + { + "budget": 0.94, + "threshold": -4.04119873046875, + "operator": "ge", + "actual_fraction": 0.942527086173495, + "metric": 0.9051445791777284, + "rescued": 2637, + "harmed": 673, + "missed_rescue": 69, + "avoided_harm": 38, + "selected_both_correct": 9261, + "selected_both_wrong": 565 + }, + { + "budget": 0.95, + "threshold": -4.1099114418029785, + "operator": "ge", + "actual_fraction": 0.9522852837769965, + "metric": 0.905503336442563, + "rescued": 2649, + "harmed": 680, + "missed_rescue": 57, + "avoided_harm": 31, + "selected_both_correct": 9376, + "selected_both_wrong": 567 + }, + { + "budget": 0.96, + "threshold": -4.188605308532715, + "operator": "ge", + "actual_fraction": 0.9628327473631341, + "metric": 0.9054315849895961, + "rescued": 2657, + "harmed": 689, + "missed_rescue": 49, + "avoided_harm": 22, + "selected_both_correct": 9503, + "selected_both_wrong": 570 + }, + { + "budget": 0.97, + "threshold": -4.283358573913574, + "operator": "ge", + "actual_fraction": 0.9731649565903709, + "metric": 0.9061490995192653, + "rescued": 2671, + "harmed": 693, + "missed_rescue": 35, + "avoided_harm": 18, + "selected_both_correct": 9628, + "selected_both_wrong": 571 + }, + { + "budget": 0.98, + "threshold": -4.372988224029541, + "operator": "ge", + "actual_fraction": 0.980483604792997, + "metric": 0.9066513596900337, + "rescued": 2684, + "harmed": 699, + "missed_rescue": 22, + "avoided_harm": 12, + "selected_both_correct": 9710, + "selected_both_wrong": 572 + }, + { + "budget": 0.99, + "threshold": -4.5110697746276855, + "operator": "ge", + "actual_fraction": 0.9896677907727631, + "metric": 0.9070818684078352, + "rescued": 2695, + "harmed": 704, + "missed_rescue": 11, + "avoided_harm": 7, + "selected_both_correct": 9819, + "selected_both_wrong": 575 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9073688742197029, + "rescued": 2706, + "harmed": 711, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9940, + "selected_both_wrong": 580 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 4888, + "rescuable": 1389, + "harmful": 388, + "both_wrong": 317, + "unit": "sample", + "total": 6982 + }, + "test": { + "both_correct": 9940, + "rescuable": 2706, + "harmful": 711, + "both_wrong": 580, + "unit": "sample", + "total": 13937 + } + }, + "small": { + "validation": 0.7556574047550845, + "test": 0.7642247255506924 + }, + "large": { + "validation": 0.8990260670295045, + "test": 0.9073688742197029 + }, + "learned_minus_confidence_test": 0.0006457630767022771, + "learned_minus_small_test": 0.1428571428571428, + "learned_minus_large_test": -0.00028700581186769103 + }, + { + "key": [ + "nlvr2", + "vilt", + "Qwen3.8" + ], + "expert_display": "ViLT", + "run": "shared_12_vilt_Qwen3.8", + "validation_rows": 6982, + "test_rows": 13937, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/vilt/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 96, + "validation_area_0_25": 0.7940561443712404, + "validation_point": { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9598968776854769, + "metric": 0.9059008879977084, + "rescued": 1446, + "harmed": 397, + "missed_rescue": 3, + "avoided_harm": 9, + "selected_both_correct": 4603, + "selected_both_wrong": 256 + }, + "test_point": { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9593169261677549, + "metric": 0.8996197172992753, + "rescued": 2770, + "harmed": 883, + "missed_rescue": 21, + "avoided_harm": 7, + "selected_both_correct": 9230, + "selected_both_wrong": 487 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7556574047550845, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1449, + "avoided_harm": 406, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5127503275871277, + "operator": "ge", + "actual_fraction": 0.009882555141793183, + "metric": 0.7588083643655114, + "rescued": 28, + "harmed": 6, + "missed_rescue": 1421, + "avoided_harm": 400, + "selected_both_correct": 28, + "selected_both_wrong": 7 + }, + { + "budget": 0.02, + "threshold": -0.5296345353126526, + "operator": "ge", + "actual_fraction": 0.01990833572042395, + "metric": 0.7610999713549126, + "rescued": 51, + "harmed": 13, + "missed_rescue": 1398, + "avoided_harm": 393, + "selected_both_correct": 62, + "selected_both_wrong": 13 + }, + { + "budget": 0.03, + "threshold": -0.5434877276420593, + "operator": "ge", + "actual_fraction": 0.029934116299054712, + "metric": 0.7632483529074764, + "rescued": 77, + "harmed": 24, + "missed_rescue": 1372, + "avoided_harm": 382, + "selected_both_correct": 91, + "selected_both_wrong": 17 + }, + { + "budget": 0.04, + "threshold": -0.56125408411026, + "operator": "ge", + "actual_fraction": 0.039959896877685475, + "metric": 0.7656831853337153, + "rescued": 102, + "harmed": 32, + "missed_rescue": 1347, + "avoided_harm": 374, + "selected_both_correct": 120, + "selected_both_wrong": 25 + }, + { + "budget": 0.05, + "threshold": -0.5779195427894592, + "operator": "ge", + "actual_fraction": 0.04998567745631624, + "metric": 0.7698367230020051, + "rescued": 132, + "harmed": 33, + "missed_rescue": 1317, + "avoided_harm": 373, + "selected_both_correct": 153, + "selected_both_wrong": 31 + }, + { + "budget": 0.06, + "threshold": -0.5911363363265991, + "operator": "ge", + "actual_fraction": 0.059868232598109423, + "metric": 0.7727012317387568, + "rescued": 156, + "harmed": 37, + "missed_rescue": 1293, + "avoided_harm": 369, + "selected_both_correct": 188, + "selected_both_wrong": 37 + }, + { + "budget": 0.07, + "threshold": -0.6087464094161987, + "operator": "ge", + "actual_fraction": 0.06989401317674018, + "metric": 0.7772844457175594, + "rescued": 191, + "harmed": 40, + "missed_rescue": 1258, + "avoided_harm": 366, + "selected_both_correct": 218, + "selected_both_wrong": 39 + }, + { + "budget": 0.08, + "threshold": -0.6274068355560303, + "operator": "ge", + "actual_fraction": 0.07991979375537095, + "metric": 0.780865081638499, + "rescued": 221, + "harmed": 45, + "missed_rescue": 1228, + "avoided_harm": 361, + "selected_both_correct": 249, + "selected_both_wrong": 43 + }, + { + "budget": 0.09, + "threshold": -0.6465112566947937, + "operator": "ge", + "actual_fraction": 0.08994557433400172, + "metric": 0.784302492122601, + "rescued": 250, + "harmed": 50, + "missed_rescue": 1199, + "avoided_harm": 356, + "selected_both_correct": 283, + "selected_both_wrong": 45 + }, + { + "budget": 0.1, + "threshold": -0.6620025038719177, + "operator": "ge", + "actual_fraction": 0.09997135491263248, + "metric": 0.7884560297908909, + "rescued": 284, + "harmed": 55, + "missed_rescue": 1165, + "avoided_harm": 351, + "selected_both_correct": 309, + "selected_both_wrong": 50 + }, + { + "budget": 0.11, + "threshold": -0.6800539493560791, + "operator": "ge", + "actual_fraction": 0.10999713549126325, + "metric": 0.7910340876539673, + "rescued": 306, + "harmed": 59, + "missed_rescue": 1143, + "avoided_harm": 347, + "selected_both_correct": 348, + "selected_both_wrong": 55 + }, + { + "budget": 0.12, + "threshold": -0.6957284212112427, + "operator": "ge", + "actual_fraction": 0.11987969063305644, + "metric": 0.7944714981380693, + "rescued": 334, + "harmed": 63, + "missed_rescue": 1115, + "avoided_harm": 343, + "selected_both_correct": 380, + "selected_both_wrong": 60 + }, + { + "budget": 0.13, + "threshold": -0.7128273248672485, + "operator": "ge", + "actual_fraction": 0.1299054712116872, + "metric": 0.7974792323116585, + "rescued": 362, + "harmed": 70, + "missed_rescue": 1087, + "avoided_harm": 336, + "selected_both_correct": 412, + "selected_both_wrong": 63 + }, + { + "budget": 0.14, + "threshold": -0.7332167625427246, + "operator": "ge", + "actual_fraction": 0.13993125179031796, + "metric": 0.8006301919220854, + "rescued": 386, + "harmed": 72, + "missed_rescue": 1063, + "avoided_harm": 334, + "selected_both_correct": 452, + "selected_both_wrong": 67 + }, + { + "budget": 0.15, + "threshold": -0.7503995895385742, + "operator": "ge", + "actual_fraction": 0.14995703236894872, + "metric": 0.8039243769693497, + "rescued": 417, + "harmed": 80, + "missed_rescue": 1032, + "avoided_harm": 326, + "selected_both_correct": 479, + "selected_both_wrong": 71 + }, + { + "budget": 0.16, + "threshold": -0.7655540704727173, + "operator": "ge", + "actual_fraction": 0.1599828129475795, + "metric": 0.8054998567745631, + "rescued": 438, + "harmed": 90, + "missed_rescue": 1011, + "avoided_harm": 316, + "selected_both_correct": 517, + "selected_both_wrong": 72 + }, + { + "budget": 0.17, + "threshold": -0.7812193036079407, + "operator": "ge", + "actual_fraction": 0.16986536808937266, + "metric": 0.8086508163849899, + "rescued": 464, + "harmed": 94, + "missed_rescue": 985, + "avoided_harm": 312, + "selected_both_correct": 551, + "selected_both_wrong": 77 + }, + { + "budget": 0.18, + "threshold": -0.7954917550086975, + "operator": "ge", + "actual_fraction": 0.17989114866800343, + "metric": 0.810369521627041, + "rescued": 481, + "harmed": 99, + "missed_rescue": 968, + "avoided_harm": 307, + "selected_both_correct": 597, + "selected_both_wrong": 79 + }, + { + "budget": 0.19, + "threshold": -0.8097342252731323, + "operator": "ge", + "actual_fraction": 0.1899169292466342, + "metric": 0.8132340303637926, + "rescued": 505, + "harmed": 103, + "missed_rescue": 944, + "avoided_harm": 303, + "selected_both_correct": 635, + "selected_both_wrong": 83 + }, + { + "budget": 0.2, + "threshold": -0.8232117891311646, + "operator": "ge", + "actual_fraction": 0.19994270982526496, + "metric": 0.8158120882268691, + "rescued": 527, + "harmed": 107, + "missed_rescue": 922, + "avoided_harm": 299, + "selected_both_correct": 673, + "selected_both_wrong": 89 + }, + { + "budget": 0.21, + "threshold": -0.8359466791152954, + "operator": "ge", + "actual_fraction": 0.20996849040389573, + "metric": 0.8181036952162704, + "rescued": 550, + "harmed": 114, + "missed_rescue": 899, + "avoided_harm": 292, + "selected_both_correct": 710, + "selected_both_wrong": 92 + }, + { + "budget": 0.22, + "threshold": -0.8473842144012451, + "operator": "ge", + "actual_fraction": 0.2199942709825265, + "metric": 0.8203953022056717, + "rescued": 574, + "harmed": 122, + "missed_rescue": 875, + "avoided_harm": 284, + "selected_both_correct": 745, + "selected_both_wrong": 95 + }, + { + "budget": 0.23, + "threshold": -0.8595863580703735, + "operator": "ge", + "actual_fraction": 0.22987682612431967, + "metric": 0.8234030363792609, + "rescued": 597, + "harmed": 124, + "missed_rescue": 852, + "avoided_harm": 282, + "selected_both_correct": 779, + "selected_both_wrong": 105 + }, + { + "budget": 0.24, + "threshold": -0.8704345226287842, + "operator": "ge", + "actual_fraction": 0.23990260670295044, + "metric": 0.8256946433686623, + "rescued": 618, + "harmed": 129, + "missed_rescue": 831, + "avoided_harm": 277, + "selected_both_correct": 819, + "selected_both_wrong": 109 + }, + { + "budget": 0.25, + "threshold": -0.8811548352241516, + "operator": "ge", + "actual_fraction": 0.2499283872815812, + "metric": 0.828845602979089, + "rescued": 647, + "harmed": 136, + "missed_rescue": 802, + "avoided_harm": 270, + "selected_both_correct": 847, + "selected_both_wrong": 115 + }, + { + "budget": 0.26, + "threshold": -0.8929013609886169, + "operator": "ge", + "actual_fraction": 0.25995416786021197, + "metric": 0.8304210827843025, + "rescued": 667, + "harmed": 145, + "missed_rescue": 782, + "avoided_harm": 261, + "selected_both_correct": 886, + "selected_both_wrong": 117 + }, + { + "budget": 0.27, + "threshold": -0.903549313545227, + "operator": "ge", + "actual_fraction": 0.26997994843884277, + "metric": 0.8327126897737038, + "rescued": 693, + "harmed": 155, + "missed_rescue": 756, + "avoided_harm": 251, + "selected_both_correct": 919, + "selected_both_wrong": 118 + }, + { + "budget": 0.28, + "threshold": -0.9115740656852722, + "operator": "ge", + "actual_fraction": 0.2798625035806359, + "metric": 0.8350042967631052, + "rescued": 716, + "harmed": 162, + "missed_rescue": 733, + "avoided_harm": 244, + "selected_both_correct": 954, + "selected_both_wrong": 122 + }, + { + "budget": 0.29, + "threshold": -0.9182595014572144, + "operator": "ge", + "actual_fraction": 0.2898882841592667, + "metric": 0.8371526783156689, + "rescued": 738, + "harmed": 169, + "missed_rescue": 711, + "avoided_harm": 237, + "selected_both_correct": 994, + "selected_both_wrong": 123 + }, + { + "budget": 0.3, + "threshold": -0.9258768558502197, + "operator": "ge", + "actual_fraction": 0.29991406473789745, + "metric": 0.8377255800630192, + "rescued": 751, + "harmed": 178, + "missed_rescue": 698, + "avoided_harm": 228, + "selected_both_correct": 1036, + "selected_both_wrong": 129 + }, + { + "budget": 0.31, + "threshold": -0.9331877827644348, + "operator": "ge", + "actual_fraction": 0.30993984531652824, + "metric": 0.8401604124892581, + "rescued": 771, + "harmed": 181, + "missed_rescue": 678, + "avoided_harm": 225, + "selected_both_correct": 1077, + "selected_both_wrong": 135 + }, + { + "budget": 0.32, + "threshold": -0.9399217367172241, + "operator": "ge", + "actual_fraction": 0.319965625895159, + "metric": 0.842595244915497, + "rescued": 793, + "harmed": 186, + "missed_rescue": 656, + "avoided_harm": 220, + "selected_both_correct": 1118, + "selected_both_wrong": 137 + }, + { + "budget": 0.33, + "threshold": -0.9453923106193542, + "operator": "ge", + "actual_fraction": 0.3299914064737898, + "metric": 0.8447436264680608, + "rescued": 814, + "harmed": 192, + "missed_rescue": 635, + "avoided_harm": 214, + "selected_both_correct": 1157, + "selected_both_wrong": 141 + }, + { + "budget": 0.34, + "threshold": -0.9515207409858704, + "operator": "ge", + "actual_fraction": 0.3398739616155829, + "metric": 0.8466055571469493, + "rescued": 830, + "harmed": 195, + "missed_rescue": 619, + "avoided_harm": 211, + "selected_both_correct": 1202, + "selected_both_wrong": 146 + }, + { + "budget": 0.35, + "threshold": -0.9567410349845886, + "operator": "ge", + "actual_fraction": 0.3498997421942137, + "metric": 0.8491836150100258, + "rescued": 854, + "harmed": 201, + "missed_rescue": 595, + "avoided_harm": 205, + "selected_both_correct": 1238, + "selected_both_wrong": 150 + }, + { + "budget": 0.36, + "threshold": -0.9606627225875854, + "operator": "ge", + "actual_fraction": 0.35992552277284445, + "metric": 0.8519048983099399, + "rescued": 876, + "harmed": 204, + "missed_rescue": 573, + "avoided_harm": 202, + "selected_both_correct": 1279, + "selected_both_wrong": 154 + }, + { + "budget": 0.37, + "threshold": -0.9650874733924866, + "operator": "ge", + "actual_fraction": 0.36995130335147525, + "metric": 0.8534803781151532, + "rescued": 894, + "harmed": 211, + "missed_rescue": 555, + "avoided_harm": 195, + "selected_both_correct": 1318, + "selected_both_wrong": 160 + }, + { + "budget": 0.38, + "threshold": -0.9686882495880127, + "operator": "ge", + "actual_fraction": 0.379977083930106, + "metric": 0.8562016614150673, + "rescued": 916, + "harmed": 214, + "missed_rescue": 533, + "avoided_harm": 192, + "selected_both_correct": 1359, + "selected_both_wrong": 164 + }, + { + "budget": 0.39, + "threshold": -0.971785306930542, + "operator": "ge", + "actual_fraction": 0.3898596390718992, + "metric": 0.8567745631624176, + "rescued": 927, + "harmed": 221, + "missed_rescue": 522, + "avoided_harm": 185, + "selected_both_correct": 1406, + "selected_both_wrong": 168 + }, + { + "budget": 0.4, + "threshold": -0.9748144745826721, + "operator": "ge", + "actual_fraction": 0.3998854196505299, + "metric": 0.859066170151819, + "rescued": 948, + "harmed": 226, + "missed_rescue": 501, + "avoided_harm": 180, + "selected_both_correct": 1447, + "selected_both_wrong": 171 + }, + { + "budget": 0.41, + "threshold": -0.977255642414093, + "operator": "ge", + "actual_fraction": 0.4099112002291607, + "metric": 0.860068748209682, + "rescued": 960, + "harmed": 231, + "missed_rescue": 489, + "avoided_harm": 175, + "selected_both_correct": 1498, + "selected_both_wrong": 173 + }, + { + "budget": 0.42, + "threshold": -0.9794421792030334, + "operator": "ge", + "actual_fraction": 0.41993698080779146, + "metric": 0.8627900315095961, + "rescued": 982, + "harmed": 234, + "missed_rescue": 467, + "avoided_harm": 172, + "selected_both_correct": 1539, + "selected_both_wrong": 177 + }, + { + "budget": 0.43, + "threshold": -0.981472373008728, + "operator": "ge", + "actual_fraction": 0.42996276138642225, + "metric": 0.8642222858779719, + "rescued": 998, + "harmed": 240, + "missed_rescue": 451, + "avoided_harm": 166, + "selected_both_correct": 1584, + "selected_both_wrong": 180 + }, + { + "budget": 0.44, + "threshold": -0.9835578799247742, + "operator": "ge", + "actual_fraction": 0.439988541965053, + "metric": 0.8653680893726726, + "rescued": 1014, + "harmed": 248, + "missed_rescue": 435, + "avoided_harm": 158, + "selected_both_correct": 1628, + "selected_both_wrong": 182 + }, + { + "budget": 0.45, + "threshold": -0.9851564168930054, + "operator": "ge", + "actual_fraction": 0.4498710971068462, + "metric": 0.8668003437410484, + "rescued": 1030, + "harmed": 254, + "missed_rescue": 419, + "avoided_harm": 152, + "selected_both_correct": 1673, + "selected_both_wrong": 184 + }, + { + "budget": 0.46, + "threshold": -0.986689031124115, + "operator": "ge", + "actual_fraction": 0.45989687768547693, + "metric": 0.8686622744199369, + "rescued": 1048, + "harmed": 259, + "missed_rescue": 401, + "avoided_harm": 147, + "selected_both_correct": 1719, + "selected_both_wrong": 185 + }, + { + "budget": 0.47, + "threshold": -0.9879001379013062, + "operator": "ge", + "actual_fraction": 0.46992265826410773, + "metric": 0.8696648524778, + "rescued": 1057, + "harmed": 261, + "missed_rescue": 392, + "avoided_harm": 145, + "selected_both_correct": 1775, + "selected_both_wrong": 188 + }, + { + "budget": 0.48, + "threshold": -0.9891784191131592, + "operator": "ge", + "actual_fraction": 0.47994843884273847, + "metric": 0.8723861357777141, + "rescued": 1077, + "harmed": 262, + "missed_rescue": 372, + "avoided_harm": 144, + "selected_both_correct": 1821, + "selected_both_wrong": 191 + }, + { + "budget": 0.49, + "threshold": -0.990114688873291, + "operator": "ge", + "actual_fraction": 0.48997421942136926, + "metric": 0.8741048410197652, + "rescued": 1092, + "harmed": 265, + "missed_rescue": 357, + "avoided_harm": 141, + "selected_both_correct": 1872, + "selected_both_wrong": 192 + }, + { + "budget": 0.5, + "threshold": -0.9910277724266052, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.8746777427671154, + "rescued": 1103, + "harmed": 272, + "missed_rescue": 346, + "avoided_harm": 134, + "selected_both_correct": 1922, + "selected_both_wrong": 194 + }, + { + "budget": 0.51, + "threshold": -0.9918718934059143, + "operator": "ge", + "actual_fraction": 0.5098825551417931, + "metric": 0.8749641936407906, + "rescued": 1110, + "harmed": 277, + "missed_rescue": 339, + "avoided_harm": 129, + "selected_both_correct": 1977, + "selected_both_wrong": 196 + }, + { + "budget": 0.52, + "threshold": -0.9926809668540955, + "operator": "ge", + "actual_fraction": 0.5199083357204239, + "metric": 0.8758235462618161, + "rescued": 1120, + "harmed": 281, + "missed_rescue": 329, + "avoided_harm": 125, + "selected_both_correct": 2033, + "selected_both_wrong": 196 + }, + { + "budget": 0.53, + "threshold": -0.9932646751403809, + "operator": "ge", + "actual_fraction": 0.5299341162990547, + "metric": 0.8762532225723288, + "rescued": 1129, + "harmed": 287, + "missed_rescue": 320, + "avoided_harm": 119, + "selected_both_correct": 2086, + "selected_both_wrong": 198 + }, + { + "budget": 0.54, + "threshold": -0.9938417673110962, + "operator": "ge", + "actual_fraction": 0.5399598968776855, + "metric": 0.8768261243196792, + "rescued": 1136, + "harmed": 290, + "missed_rescue": 313, + "avoided_harm": 116, + "selected_both_correct": 2144, + "selected_both_wrong": 200 + }, + { + "budget": 0.55, + "threshold": -0.9944412112236023, + "operator": "ge", + "actual_fraction": 0.5499856774563162, + "metric": 0.8778287023775423, + "rescued": 1146, + "harmed": 293, + "missed_rescue": 303, + "avoided_harm": 113, + "selected_both_correct": 2200, + "selected_both_wrong": 201 + }, + { + "budget": 0.56, + "threshold": -0.9949513673782349, + "operator": "ge", + "actual_fraction": 0.5598682325981095, + "metric": 0.878258378688055, + "rescued": 1156, + "harmed": 300, + "missed_rescue": 293, + "avoided_harm": 106, + "selected_both_correct": 2249, + "selected_both_wrong": 204 + }, + { + "budget": 0.57, + "threshold": -0.9954326152801514, + "operator": "ge", + "actual_fraction": 0.5698940131767402, + "metric": 0.8792609567459181, + "rescued": 1170, + "harmed": 307, + "missed_rescue": 279, + "avoided_harm": 99, + "selected_both_correct": 2295, + "selected_both_wrong": 207 + }, + { + "budget": 0.58, + "threshold": -0.9958608746528625, + "operator": "ge", + "actual_fraction": 0.579919793755371, + "metric": 0.8805499856774563, + "rescued": 1182, + "harmed": 310, + "missed_rescue": 267, + "avoided_harm": 96, + "selected_both_correct": 2347, + "selected_both_wrong": 210 + }, + { + "budget": 0.59, + "threshold": -0.9962128400802612, + "operator": "ge", + "actual_fraction": 0.5899455743340017, + "metric": 0.8819822400458321, + "rescued": 1195, + "harmed": 313, + "missed_rescue": 254, + "avoided_harm": 93, + "selected_both_correct": 2399, + "selected_both_wrong": 212 + }, + { + "budget": 0.6, + "threshold": -0.9965764880180359, + "operator": "ge", + "actual_fraction": 0.5999713549126325, + "metric": 0.8819822400458321, + "rescued": 1202, + "harmed": 320, + "missed_rescue": 247, + "avoided_harm": 86, + "selected_both_correct": 2454, + "selected_both_wrong": 213 + }, + { + "budget": 0.61, + "threshold": -0.9968799352645874, + "operator": "ge", + "actual_fraction": 0.6099971354912632, + "metric": 0.8832712689773704, + "rescued": 1214, + "harmed": 323, + "missed_rescue": 235, + "avoided_harm": 83, + "selected_both_correct": 2507, + "selected_both_wrong": 215 + }, + { + "budget": 0.62, + "threshold": -0.9971383810043335, + "operator": "ge", + "actual_fraction": 0.6198796906330565, + "metric": 0.8832712689773704, + "rescued": 1218, + "harmed": 327, + "missed_rescue": 231, + "avoided_harm": 79, + "selected_both_correct": 2566, + "selected_both_wrong": 217 + }, + { + "budget": 0.63, + "threshold": -0.997377872467041, + "operator": "ge", + "actual_fraction": 0.6299054712116872, + "metric": 0.8842738470352335, + "rescued": 1228, + "harmed": 330, + "missed_rescue": 221, + "avoided_harm": 76, + "selected_both_correct": 2619, + "selected_both_wrong": 221 + }, + { + "budget": 0.64, + "threshold": -0.9976021647453308, + "operator": "ge", + "actual_fraction": 0.639931251790318, + "metric": 0.886135777714122, + "rescued": 1243, + "harmed": 332, + "missed_rescue": 206, + "avoided_harm": 74, + "selected_both_correct": 2672, + "selected_both_wrong": 221 + }, + { + "budget": 0.65, + "threshold": -0.9978214502334595, + "operator": "ge", + "actual_fraction": 0.6499570323689488, + "metric": 0.8867086794614724, + "rescued": 1249, + "harmed": 334, + "missed_rescue": 200, + "avoided_harm": 72, + "selected_both_correct": 2732, + "selected_both_wrong": 223 + }, + { + "budget": 0.66, + "threshold": -0.9980514049530029, + "operator": "ge", + "actual_fraction": 0.6599828129475795, + "metric": 0.8882841592666858, + "rescued": 1261, + "harmed": 335, + "missed_rescue": 188, + "avoided_harm": 71, + "selected_both_correct": 2789, + "selected_both_wrong": 223 + }, + { + "budget": 0.67, + "threshold": -0.9982016086578369, + "operator": "ge", + "actual_fraction": 0.6698653680893727, + "metric": 0.889573188198224, + "rescued": 1271, + "harmed": 336, + "missed_rescue": 178, + "avoided_harm": 70, + "selected_both_correct": 2847, + "selected_both_wrong": 223 + }, + { + "budget": 0.68, + "threshold": -0.9983693957328796, + "operator": "ge", + "actual_fraction": 0.6798911486680035, + "metric": 0.8907189916929247, + "rescued": 1280, + "harmed": 337, + "missed_rescue": 169, + "avoided_harm": 69, + "selected_both_correct": 2907, + "selected_both_wrong": 223 + }, + { + "budget": 0.69, + "threshold": -0.9984915256500244, + "operator": "ge", + "actual_fraction": 0.6899169292466342, + "metric": 0.8921512460613005, + "rescued": 1293, + "harmed": 340, + "missed_rescue": 156, + "avoided_harm": 66, + "selected_both_correct": 2960, + "selected_both_wrong": 224 + }, + { + "budget": 0.7, + "threshold": -0.9986109733581543, + "operator": "gt", + "actual_fraction": 0.6997994843884274, + "metric": 0.8928673732454884, + "rescued": 1299, + "harmed": 341, + "missed_rescue": 150, + "avoided_harm": 65, + "selected_both_correct": 3021, + "selected_both_wrong": 225 + }, + { + "budget": 0.71, + "threshold": -0.9986966252326965, + "operator": "ge", + "actual_fraction": 0.7099684904038958, + "metric": 0.8931538241191636, + "rescued": 1304, + "harmed": 344, + "missed_rescue": 145, + "avoided_harm": 62, + "selected_both_correct": 3084, + "selected_both_wrong": 225 + }, + { + "budget": 0.72, + "threshold": -0.9987963438034058, + "operator": "ge", + "actual_fraction": 0.7199942709825266, + "metric": 0.8935835004296763, + "rescued": 1310, + "harmed": 347, + "missed_rescue": 139, + "avoided_harm": 59, + "selected_both_correct": 3144, + "selected_both_wrong": 226 + }, + { + "budget": 0.73, + "threshold": -0.998875081539154, + "operator": "ge", + "actual_fraction": 0.7298768261243197, + "metric": 0.8940131767401891, + "rescued": 1318, + "harmed": 352, + "missed_rescue": 131, + "avoided_harm": 54, + "selected_both_correct": 3199, + "selected_both_wrong": 227 + }, + { + "budget": 0.74, + "threshold": -0.9989551305770874, + "operator": "ge", + "actual_fraction": 0.7399026067029505, + "metric": 0.8951589802348897, + "rescued": 1327, + "harmed": 353, + "missed_rescue": 122, + "avoided_harm": 53, + "selected_both_correct": 3257, + "selected_both_wrong": 229 + }, + { + "budget": 0.75, + "threshold": -0.9990480542182922, + "operator": "ge", + "actual_fraction": 0.7499283872815812, + "metric": 0.8961615582927528, + "rescued": 1336, + "harmed": 355, + "missed_rescue": 113, + "avoided_harm": 51, + "selected_both_correct": 3314, + "selected_both_wrong": 231 + }, + { + "budget": 0.76, + "threshold": -0.999126136302948, + "operator": "ge", + "actual_fraction": 0.759954167860212, + "metric": 0.8973073617874534, + "rescued": 1345, + "harmed": 356, + "missed_rescue": 104, + "avoided_harm": 50, + "selected_both_correct": 3374, + "selected_both_wrong": 231 + }, + { + "budget": 0.77, + "threshold": -0.999184787273407, + "operator": "gt", + "actual_fraction": 0.7698367230020051, + "metric": 0.8977370380979662, + "rescued": 1350, + "harmed": 358, + "missed_rescue": 99, + "avoided_harm": 48, + "selected_both_correct": 3435, + "selected_both_wrong": 232 + }, + { + "budget": 0.78, + "threshold": -0.9992268085479736, + "operator": "ge", + "actual_fraction": 0.7798625035806359, + "metric": 0.8987396161558293, + "rescued": 1359, + "harmed": 360, + "missed_rescue": 90, + "avoided_harm": 46, + "selected_both_correct": 3492, + "selected_both_wrong": 234 + }, + { + "budget": 0.79, + "threshold": -0.9992831349372864, + "operator": "ge", + "actual_fraction": 0.7898882841592667, + "metric": 0.8997421942136924, + "rescued": 1370, + "harmed": 364, + "missed_rescue": 79, + "avoided_harm": 42, + "selected_both_correct": 3545, + "selected_both_wrong": 236 + }, + { + "budget": 0.8, + "threshold": -0.9993318915367126, + "operator": "ge", + "actual_fraction": 0.7999140647378975, + "metric": 0.8997421942136924, + "rescued": 1372, + "harmed": 366, + "missed_rescue": 77, + "avoided_harm": 40, + "selected_both_correct": 3611, + "selected_both_wrong": 236 + }, + { + "budget": 0.81, + "threshold": -0.9993773102760315, + "operator": "ge", + "actual_fraction": 0.8099398453165282, + "metric": 0.9000286450873675, + "rescued": 1376, + "harmed": 368, + "missed_rescue": 73, + "avoided_harm": 38, + "selected_both_correct": 3674, + "selected_both_wrong": 237 + }, + { + "budget": 0.82, + "threshold": -0.9994175434112549, + "operator": "ge", + "actual_fraction": 0.819965625895159, + "metric": 0.9003150959610426, + "rescued": 1381, + "harmed": 371, + "missed_rescue": 68, + "avoided_harm": 35, + "selected_both_correct": 3734, + "selected_both_wrong": 239 + }, + { + "budget": 0.83, + "threshold": -0.999452531337738, + "operator": "ge", + "actual_fraction": 0.8299914064737898, + "metric": 0.9003150959610426, + "rescued": 1383, + "harmed": 373, + "missed_rescue": 66, + "avoided_harm": 33, + "selected_both_correct": 3798, + "selected_both_wrong": 241 + }, + { + "budget": 0.84, + "threshold": -0.9994945526123047, + "operator": "ge", + "actual_fraction": 0.8398739616155829, + "metric": 0.9011744485820682, + "rescued": 1389, + "harmed": 373, + "missed_rescue": 60, + "avoided_harm": 33, + "selected_both_correct": 3861, + "selected_both_wrong": 241 + }, + { + "budget": 0.85, + "threshold": -0.9995274543762207, + "operator": "gt", + "actual_fraction": 0.8497565167573761, + "metric": 0.9016041248925809, + "rescued": 1394, + "harmed": 375, + "missed_rescue": 55, + "avoided_harm": 31, + "selected_both_correct": 3922, + "selected_both_wrong": 242 + }, + { + "budget": 0.86, + "threshold": -0.9995617270469666, + "operator": "ge", + "actual_fraction": 0.8599255227728445, + "metric": 0.902606702950444, + "rescued": 1401, + "harmed": 375, + "missed_rescue": 48, + "avoided_harm": 31, + "selected_both_correct": 3985, + "selected_both_wrong": 243 + }, + { + "budget": 0.87, + "threshold": -0.9995949864387512, + "operator": "ge", + "actual_fraction": 0.8699513033514752, + "metric": 0.9027499283872816, + "rescued": 1407, + "harmed": 380, + "missed_rescue": 42, + "avoided_harm": 26, + "selected_both_correct": 4043, + "selected_both_wrong": 244 + }, + { + "budget": 0.88, + "threshold": -0.9996248483657837, + "operator": "ge", + "actual_fraction": 0.879977083930106, + "metric": 0.9031796046977943, + "rescued": 1412, + "harmed": 382, + "missed_rescue": 37, + "avoided_harm": 24, + "selected_both_correct": 4106, + "selected_both_wrong": 244 + }, + { + "budget": 0.89, + "threshold": -0.9996511936187744, + "operator": "ge", + "actual_fraction": 0.8898596390718991, + "metric": 0.9034660555714695, + "rescued": 1419, + "harmed": 387, + "missed_rescue": 30, + "avoided_harm": 19, + "selected_both_correct": 4161, + "selected_both_wrong": 246 + }, + { + "budget": 0.9, + "threshold": -0.9996830224990845, + "operator": "ge", + "actual_fraction": 0.8998854196505299, + "metric": 0.9038957318819822, + "rescued": 1423, + "harmed": 388, + "missed_rescue": 26, + "avoided_harm": 18, + "selected_both_correct": 4225, + "selected_both_wrong": 247 + }, + { + "budget": 0.91, + "threshold": -0.9997097849845886, + "operator": "gt", + "actual_fraction": 0.9097679747923231, + "metric": 0.9046118590661701, + "rescued": 1431, + "harmed": 391, + "missed_rescue": 18, + "avoided_harm": 15, + "selected_both_correct": 4283, + "selected_both_wrong": 247 + }, + { + "budget": 0.92, + "threshold": -0.9997391104698181, + "operator": "ge", + "actual_fraction": 0.9199369808077915, + "metric": 0.9050415353766829, + "rescued": 1434, + "harmed": 391, + "missed_rescue": 15, + "avoided_harm": 15, + "selected_both_correct": 4350, + "selected_both_wrong": 248 + }, + { + "budget": 0.93, + "threshold": -0.9997641444206238, + "operator": "ge", + "actual_fraction": 0.9299627613864222, + "metric": 0.9051847608135205, + "rescued": 1437, + "harmed": 393, + "missed_rescue": 12, + "avoided_harm": 13, + "selected_both_correct": 4411, + "selected_both_wrong": 252 + }, + { + "budget": 0.94, + "threshold": -0.9997876286506653, + "operator": "ge", + "actual_fraction": 0.939988541965053, + "metric": 0.905327986250358, + "rescued": 1439, + "harmed": 394, + "missed_rescue": 10, + "avoided_harm": 12, + "selected_both_correct": 4476, + "selected_both_wrong": 254 + }, + { + "budget": 0.95, + "threshold": -0.9998131394386292, + "operator": "ge", + "actual_fraction": 0.9498710971068461, + "metric": 0.9056144371240332, + "rescued": 1442, + "harmed": 395, + "missed_rescue": 7, + "avoided_harm": 11, + "selected_both_correct": 4540, + "selected_both_wrong": 255 + }, + { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9598968776854769, + "metric": 0.9059008879977084, + "rescued": 1446, + "harmed": 397, + "missed_rescue": 3, + "avoided_harm": 9, + "selected_both_correct": 4603, + "selected_both_wrong": 256 + }, + { + "budget": 0.97, + "threshold": -0.999868631362915, + "operator": "ge", + "actual_fraction": 0.9699226582641077, + "metric": 0.9054712116871957, + "rescued": 1446, + "harmed": 400, + "missed_rescue": 3, + "avoided_harm": 6, + "selected_both_correct": 4670, + "selected_both_wrong": 256 + }, + { + "budget": 0.98, + "threshold": -0.9998925924301147, + "operator": "ge", + "actual_fraction": 0.9799484388427385, + "metric": 0.9051847608135205, + "rescued": 1446, + "harmed": 402, + "missed_rescue": 3, + "avoided_harm": 4, + "selected_both_correct": 4738, + "selected_both_wrong": 256 + }, + { + "budget": 0.99, + "threshold": -0.99992835521698, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.9047550845030078, + "rescued": 1446, + "harmed": 405, + "missed_rescue": 3, + "avoided_harm": 1, + "selected_both_correct": 4805, + "selected_both_wrong": 256 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9050415353766829, + "rescued": 1449, + "harmed": 406, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 4870, + "selected_both_wrong": 257 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7642247255506924, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2791, + "avoided_harm": 890, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5127503275871277, + "operator": "ge", + "actual_fraction": 0.007749156920427639, + "metric": 0.7660185118748655, + "rescued": 36, + "harmed": 11, + "missed_rescue": 2755, + "avoided_harm": 879, + "selected_both_correct": 57, + "selected_both_wrong": 4 + }, + { + "budget": 0.02, + "threshold": -0.5296345353126526, + "operator": "ge", + "actual_fraction": 0.017650857429862953, + "metric": 0.7696778359761786, + "rescued": 96, + "harmed": 20, + "missed_rescue": 2695, + "avoided_harm": 870, + "selected_both_correct": 120, + "selected_both_wrong": 10 + }, + { + "budget": 0.03, + "threshold": -0.5434877276420593, + "operator": "ge", + "actual_fraction": 0.025113008538422903, + "metric": 0.7726196455478224, + "rescued": 144, + "harmed": 27, + "missed_rescue": 2647, + "avoided_harm": 863, + "selected_both_correct": 161, + "selected_both_wrong": 18 + }, + { + "budget": 0.04, + "threshold": -0.56125408411026, + "operator": "ge", + "actual_fraction": 0.0349429575948913, + "metric": 0.776565975461003, + "rescued": 205, + "harmed": 33, + "missed_rescue": 2586, + "avoided_harm": 857, + "selected_both_correct": 226, + "selected_both_wrong": 23 + }, + { + "budget": 0.05, + "threshold": -0.5779195427894592, + "operator": "ge", + "actual_fraction": 0.043840137762789694, + "metric": 0.7785750161440769, + "rescued": 242, + "harmed": 42, + "missed_rescue": 2549, + "avoided_harm": 848, + "selected_both_correct": 291, + "selected_both_wrong": 36 + }, + { + "budget": 0.06, + "threshold": -0.5911363363265991, + "operator": "ge", + "actual_fraction": 0.05187630049508503, + "metric": 0.7809428140919854, + "rescued": 286, + "harmed": 53, + "missed_rescue": 2505, + "avoided_harm": 837, + "selected_both_correct": 345, + "selected_both_wrong": 39 + }, + { + "budget": 0.07, + "threshold": -0.6087464094161987, + "operator": "ge", + "actual_fraction": 0.06177800100452034, + "metric": 0.7835976178517615, + "rescued": 336, + "harmed": 66, + "missed_rescue": 2455, + "avoided_harm": 824, + "selected_both_correct": 409, + "selected_both_wrong": 50 + }, + { + "budget": 0.08, + "threshold": -0.6274068355560303, + "operator": "ge", + "actual_fraction": 0.07132094424912104, + "metric": 0.7864676759704384, + "rescued": 387, + "harmed": 77, + "missed_rescue": 2404, + "avoided_harm": 813, + "selected_both_correct": 473, + "selected_both_wrong": 57 + }, + { + "budget": 0.09, + "threshold": -0.6465112566947937, + "operator": "ge", + "actual_fraction": 0.08251417091196096, + "metric": 0.7888354739183469, + "rescued": 439, + "harmed": 96, + "missed_rescue": 2352, + "avoided_harm": 794, + "selected_both_correct": 546, + "selected_both_wrong": 69 + }, + { + "budget": 0.1, + "threshold": -0.6620025038719177, + "operator": "ge", + "actual_fraction": 0.09090909090909091, + "metric": 0.7910597689603215, + "rescued": 483, + "harmed": 109, + "missed_rescue": 2308, + "avoided_harm": 781, + "selected_both_correct": 601, + "selected_both_wrong": 74 + }, + { + "budget": 0.11, + "threshold": -0.6800539493560791, + "operator": "ge", + "actual_fraction": 0.10030853124775777, + "metric": 0.7934275669082299, + "rescued": 529, + "harmed": 122, + "missed_rescue": 2262, + "avoided_harm": 768, + "selected_both_correct": 661, + "selected_both_wrong": 86 + }, + { + "budget": 0.12, + "threshold": -0.6957284212112427, + "operator": "ge", + "actual_fraction": 0.11078424338092846, + "metric": 0.7971586424625099, + "rescued": 592, + "harmed": 133, + "missed_rescue": 2199, + "avoided_harm": 757, + "selected_both_correct": 723, + "selected_both_wrong": 96 + }, + { + "budget": 0.13, + "threshold": -0.7128273248672485, + "operator": "ge", + "actual_fraction": 0.12075769534333071, + "metric": 0.7997416947693191, + "rescued": 640, + "harmed": 145, + "missed_rescue": 2151, + "avoided_harm": 745, + "selected_both_correct": 792, + "selected_both_wrong": 106 + }, + { + "budget": 0.14, + "threshold": -0.7332167625427246, + "operator": "ge", + "actual_fraction": 0.13431871995407907, + "metric": 0.8044055392121691, + "rescued": 720, + "harmed": 160, + "missed_rescue": 2071, + "avoided_harm": 730, + "selected_both_correct": 870, + "selected_both_wrong": 122 + }, + { + "budget": 0.15, + "threshold": -0.7503995895385742, + "operator": "ge", + "actual_fraction": 0.144579177728349, + "metric": 0.8072038458778791, + "rescued": 769, + "harmed": 170, + "missed_rescue": 2022, + "avoided_harm": 720, + "selected_both_correct": 947, + "selected_both_wrong": 129 + }, + { + "budget": 0.16, + "threshold": -0.7655540704727173, + "operator": "ge", + "actual_fraction": 0.1534763578962474, + "metric": 0.8099304010906221, + "rescued": 817, + "harmed": 180, + "missed_rescue": 1974, + "avoided_harm": 710, + "selected_both_correct": 1002, + "selected_both_wrong": 140 + }, + { + "budget": 0.17, + "threshold": -0.7812193036079407, + "operator": "ge", + "actual_fraction": 0.16280404678194735, + "metric": 0.8125852048503982, + "rescued": 868, + "harmed": 194, + "missed_rescue": 1923, + "avoided_harm": 696, + "selected_both_correct": 1059, + "selected_both_wrong": 148 + }, + { + "budget": 0.18, + "threshold": -0.7954917550086975, + "operator": "ge", + "actual_fraction": 0.17306450455621727, + "metric": 0.8159575231398436, + "rescued": 927, + "harmed": 206, + "missed_rescue": 1864, + "avoided_harm": 684, + "selected_both_correct": 1124, + "selected_both_wrong": 155 + }, + { + "budget": 0.19, + "threshold": -0.8097342252731323, + "operator": "ge", + "actual_fraction": 0.18196168472411567, + "metric": 0.8179665638229174, + "rescued": 966, + "harmed": 217, + "missed_rescue": 1825, + "avoided_harm": 673, + "selected_both_correct": 1191, + "selected_both_wrong": 162 + }, + { + "budget": 0.2, + "threshold": -0.8232117891311646, + "operator": "ge", + "actual_fraction": 0.1928679055750879, + "metric": 0.820262610317859, + "rescued": 1011, + "harmed": 230, + "missed_rescue": 1780, + "avoided_harm": 660, + "selected_both_correct": 1276, + "selected_both_wrong": 171 + }, + { + "budget": 0.21, + "threshold": -0.8359466791152954, + "operator": "ge", + "actual_fraction": 0.20104757121331707, + "metric": 0.8221998995479658, + "rescued": 1049, + "harmed": 241, + "missed_rescue": 1742, + "avoided_harm": 649, + "selected_both_correct": 1333, + "selected_both_wrong": 179 + }, + { + "budget": 0.22, + "threshold": -0.8473842144012451, + "operator": "ge", + "actual_fraction": 0.21051876300495084, + "metric": 0.8254287149314774, + "rescued": 1098, + "harmed": 245, + "missed_rescue": 1693, + "avoided_harm": 645, + "selected_both_correct": 1403, + "selected_both_wrong": 188 + }, + { + "budget": 0.23, + "threshold": -0.8595863580703735, + "operator": "ge", + "actual_fraction": 0.21998995479658462, + "metric": 0.8272225012556504, + "rescued": 1135, + "harmed": 257, + "missed_rescue": 1656, + "avoided_harm": 633, + "selected_both_correct": 1478, + "selected_both_wrong": 196 + }, + { + "budget": 0.24, + "threshold": -0.8704345226287842, + "operator": "ge", + "actual_fraction": 0.22838487479371458, + "metric": 0.8280117672382866, + "rescued": 1161, + "harmed": 272, + "missed_rescue": 1630, + "avoided_harm": 618, + "selected_both_correct": 1544, + "selected_both_wrong": 206 + }, + { + "budget": 0.25, + "threshold": -0.8811548352241516, + "operator": "ge", + "actual_fraction": 0.23900408983281912, + "metric": 0.8303078137332282, + "rescued": 1207, + "harmed": 286, + "missed_rescue": 1584, + "avoided_harm": 604, + "selected_both_correct": 1623, + "selected_both_wrong": 215 + }, + { + "budget": 0.26, + "threshold": -0.8929013609886169, + "operator": "ge", + "actual_fraction": 0.2506995766664275, + "metric": 0.8333931262108057, + "rescued": 1260, + "harmed": 296, + "missed_rescue": 1531, + "avoided_harm": 594, + "selected_both_correct": 1712, + "selected_both_wrong": 226 + }, + { + "budget": 0.27, + "threshold": -0.903549313545227, + "operator": "ge", + "actual_fraction": 0.2646911099949774, + "metric": 0.8365501901413503, + "rescued": 1318, + "harmed": 310, + "missed_rescue": 1473, + "avoided_harm": 580, + "selected_both_correct": 1828, + "selected_both_wrong": 233 + }, + { + "budget": 0.28, + "threshold": -0.9115740656852722, + "operator": "ge", + "actual_fraction": 0.27437755614551196, + "metric": 0.8389897395422258, + "rescued": 1366, + "harmed": 324, + "missed_rescue": 1425, + "avoided_harm": 566, + "selected_both_correct": 1899, + "selected_both_wrong": 235 + }, + { + "budget": 0.29, + "threshold": -0.9182595014572144, + "operator": "ge", + "actual_fraction": 0.2819832101600057, + "metric": 0.8409987802252996, + "rescued": 1401, + "harmed": 331, + "missed_rescue": 1390, + "avoided_harm": 559, + "selected_both_correct": 1955, + "selected_both_wrong": 243 + }, + { + "budget": 0.3, + "threshold": -0.9258768558502197, + "operator": "ge", + "actual_fraction": 0.2907368874219703, + "metric": 0.8434383296261749, + "rescued": 1446, + "harmed": 342, + "missed_rescue": 1345, + "avoided_harm": 548, + "selected_both_correct": 2015, + "selected_both_wrong": 249 + }, + { + "budget": 0.31, + "threshold": -0.9331877827644348, + "operator": "ge", + "actual_fraction": 0.302145368443711, + "metric": 0.8450168615914472, + "rescued": 1488, + "harmed": 362, + "missed_rescue": 1303, + "avoided_harm": 528, + "selected_both_correct": 2106, + "selected_both_wrong": 255 + }, + { + "budget": 0.32, + "threshold": -0.9399217367172241, + "operator": "ge", + "actual_fraction": 0.31412786108918705, + "metric": 0.847025902274521, + "rescued": 1531, + "harmed": 377, + "missed_rescue": 1260, + "avoided_harm": 513, + "selected_both_correct": 2209, + "selected_both_wrong": 261 + }, + { + "budget": 0.33, + "threshold": -0.9453923106193542, + "operator": "ge", + "actual_fraction": 0.3238143072397216, + "metric": 0.8486761856927603, + "rescued": 1564, + "harmed": 387, + "missed_rescue": 1227, + "avoided_harm": 503, + "selected_both_correct": 2293, + "selected_both_wrong": 269 + }, + { + "budget": 0.34, + "threshold": -0.9515207409858704, + "operator": "ge", + "actual_fraction": 0.33529453971442924, + "metric": 0.8514744923584703, + "rescued": 1615, + "harmed": 399, + "missed_rescue": 1176, + "avoided_harm": 491, + "selected_both_correct": 2385, + "selected_both_wrong": 274 + }, + { + "budget": 0.35, + "threshold": -0.9567410349845886, + "operator": "ge", + "actual_fraction": 0.3452679916768315, + "metric": 0.8531247757767095, + "rescued": 1654, + "harmed": 415, + "missed_rescue": 1137, + "avoided_harm": 475, + "selected_both_correct": 2465, + "selected_both_wrong": 278 + }, + { + "budget": 0.36, + "threshold": -0.9606627225875854, + "operator": "ge", + "actual_fraction": 0.35595895816890294, + "metric": 0.855420822271651, + "rescued": 1697, + "harmed": 426, + "missed_rescue": 1094, + "avoided_harm": 464, + "selected_both_correct": 2554, + "selected_both_wrong": 284 + }, + { + "budget": 0.37, + "threshold": -0.9650874733924866, + "operator": "ge", + "actual_fraction": 0.36837195953218055, + "metric": 0.8573581115017579, + "rescued": 1745, + "harmed": 447, + "missed_rescue": 1046, + "avoided_harm": 443, + "selected_both_correct": 2651, + "selected_both_wrong": 291 + }, + { + "budget": 0.38, + "threshold": -0.9686882495880127, + "operator": "ge", + "actual_fraction": 0.3797086891009543, + "metric": 0.8592954007318648, + "rescued": 1785, + "harmed": 460, + "missed_rescue": 1006, + "avoided_harm": 430, + "selected_both_correct": 2747, + "selected_both_wrong": 300 + }, + { + "budget": 0.39, + "threshold": -0.971785306930542, + "operator": "ge", + "actual_fraction": 0.391117170122695, + "metric": 0.8620219559446078, + "rescued": 1832, + "harmed": 469, + "missed_rescue": 959, + "avoided_harm": 421, + "selected_both_correct": 2845, + "selected_both_wrong": 305 + }, + { + "budget": 0.4, + "threshold": -0.9748144745826721, + "operator": "ge", + "actual_fraction": 0.40231039678553493, + "metric": 0.8641744995336156, + "rescued": 1873, + "harmed": 480, + "missed_rescue": 918, + "avoided_harm": 410, + "selected_both_correct": 2945, + "selected_both_wrong": 309 + }, + { + "budget": 0.41, + "threshold": -0.977255642414093, + "operator": "ge", + "actual_fraction": 0.41235560020090406, + "metric": 0.8662552916696563, + "rescued": 1912, + "harmed": 490, + "missed_rescue": 879, + "avoided_harm": 400, + "selected_both_correct": 3027, + "selected_both_wrong": 318 + }, + { + "budget": 0.42, + "threshold": -0.9794421792030334, + "operator": "ge", + "actual_fraction": 0.4231183181459425, + "metric": 0.8680490779938294, + "rescued": 1950, + "harmed": 503, + "missed_rescue": 841, + "avoided_harm": 387, + "selected_both_correct": 3119, + "selected_both_wrong": 325 + }, + { + "budget": 0.43, + "threshold": -0.981472373008728, + "operator": "ge", + "actual_fraction": 0.43172849250197315, + "metric": 0.869268852694267, + "rescued": 1972, + "harmed": 508, + "missed_rescue": 819, + "avoided_harm": 382, + "selected_both_correct": 3207, + "selected_both_wrong": 330 + }, + { + "budget": 0.44, + "threshold": -0.9835578799247742, + "operator": "ge", + "actual_fraction": 0.4432804764296477, + "metric": 0.8709191361125063, + "rescued": 2005, + "harmed": 518, + "missed_rescue": 786, + "avoided_harm": 372, + "selected_both_correct": 3320, + "selected_both_wrong": 335 + }, + { + "budget": 0.45, + "threshold": -0.9851564168930054, + "operator": "ge", + "actual_fraction": 0.4547607089043553, + "metric": 0.8727129224366793, + "rescued": 2042, + "harmed": 530, + "missed_rescue": 749, + "avoided_harm": 360, + "selected_both_correct": 3422, + "selected_both_wrong": 344 + }, + { + "budget": 0.46, + "threshold": -0.986689031124115, + "operator": "ge", + "actual_fraction": 0.4649494152256583, + "metric": 0.8740762000430509, + "rescued": 2074, + "harmed": 543, + "missed_rescue": 717, + "avoided_harm": 347, + "selected_both_correct": 3513, + "selected_both_wrong": 350 + }, + { + "budget": 0.47, + "threshold": -0.9879001379013062, + "operator": "ge", + "actual_fraction": 0.47399009829949057, + "metric": 0.8749372174786539, + "rescued": 2096, + "harmed": 553, + "missed_rescue": 695, + "avoided_harm": 337, + "selected_both_correct": 3603, + "selected_both_wrong": 354 + }, + { + "budget": 0.48, + "threshold": -0.9891784191131592, + "operator": "ge", + "actual_fraction": 0.48446581043266124, + "metric": 0.8764439979909593, + "rescued": 2126, + "harmed": 562, + "missed_rescue": 665, + "avoided_harm": 328, + "selected_both_correct": 3704, + "selected_both_wrong": 360 + }, + { + "budget": 0.49, + "threshold": -0.990114688873291, + "operator": "ge", + "actual_fraction": 0.491210447011552, + "metric": 0.8771615125206286, + "rescued": 2145, + "harmed": 571, + "missed_rescue": 646, + "avoided_harm": 319, + "selected_both_correct": 3767, + "selected_both_wrong": 363 + }, + { + "budget": 0.5, + "threshold": -0.9910277724266052, + "operator": "ge", + "actual_fraction": 0.5008251417091196, + "metric": 0.8780942814091985, + "rescued": 2168, + "harmed": 581, + "missed_rescue": 623, + "avoided_harm": 309, + "selected_both_correct": 3859, + "selected_both_wrong": 372 + }, + { + "budget": 0.51, + "threshold": -0.9918718934059143, + "operator": "ge", + "actual_fraction": 0.5107268422185549, + "metric": 0.878596541579967, + "rescued": 2183, + "harmed": 589, + "missed_rescue": 608, + "avoided_harm": 301, + "selected_both_correct": 3969, + "selected_both_wrong": 377 + }, + { + "budget": 0.52, + "threshold": -0.9926809668540955, + "operator": "ge", + "actual_fraction": 0.5217048145224941, + "metric": 0.8796010619215039, + "rescued": 2208, + "harmed": 600, + "missed_rescue": 583, + "avoided_harm": 290, + "selected_both_correct": 4083, + "selected_both_wrong": 380 + }, + { + "budget": 0.53, + "threshold": -0.9932646751403809, + "operator": "ge", + "actual_fraction": 0.5306737461433594, + "metric": 0.8807490851689747, + "rescued": 2231, + "harmed": 607, + "missed_rescue": 560, + "avoided_harm": 283, + "selected_both_correct": 4172, + "selected_both_wrong": 386 + }, + { + "budget": 0.54, + "threshold": -0.9938417673110962, + "operator": "ge", + "actual_fraction": 0.5398579321231255, + "metric": 0.8816101026045777, + "rescued": 2253, + "harmed": 617, + "missed_rescue": 538, + "avoided_harm": 273, + "selected_both_correct": 4267, + "selected_both_wrong": 387 + }, + { + "budget": 0.55, + "threshold": -0.9944412112236023, + "operator": "ge", + "actual_fraction": 0.5504771471622301, + "metric": 0.882327617134247, + "rescued": 2273, + "harmed": 627, + "missed_rescue": 518, + "avoided_harm": 263, + "selected_both_correct": 4379, + "selected_both_wrong": 393 + }, + { + "budget": 0.56, + "threshold": -0.9949513673782349, + "operator": "ge", + "actual_fraction": 0.5604505991246322, + "metric": 0.8834756403817178, + "rescued": 2298, + "harmed": 636, + "missed_rescue": 493, + "avoided_harm": 254, + "selected_both_correct": 4479, + "selected_both_wrong": 398 + }, + { + "budget": 0.57, + "threshold": -0.9954326152801514, + "operator": "ge", + "actual_fraction": 0.570926311257803, + "metric": 0.8845519121762215, + "rescued": 2321, + "harmed": 644, + "missed_rescue": 470, + "avoided_harm": 246, + "selected_both_correct": 4592, + "selected_both_wrong": 400 + }, + { + "budget": 0.58, + "threshold": -0.9958608746528625, + "operator": "ge", + "actual_fraction": 0.5816172777498745, + "metric": 0.8858434383296262, + "rescued": 2349, + "harmed": 654, + "missed_rescue": 442, + "avoided_harm": 236, + "selected_both_correct": 4698, + "selected_both_wrong": 405 + }, + { + "budget": 0.59, + "threshold": -0.9962128400802612, + "operator": "ge", + "actual_fraction": 0.5920929898830452, + "metric": 0.8866327043122624, + "rescued": 2374, + "harmed": 668, + "missed_rescue": 417, + "avoided_harm": 222, + "selected_both_correct": 4804, + "selected_both_wrong": 406 + }, + { + "budget": 0.6, + "threshold": -0.9965764880180359, + "operator": "ge", + "actual_fraction": 0.6034297194518189, + "metric": 0.8874219702948984, + "rescued": 2395, + "harmed": 678, + "missed_rescue": 396, + "avoided_harm": 212, + "selected_both_correct": 4930, + "selected_both_wrong": 407 + }, + { + "budget": 0.61, + "threshold": -0.9968799352645874, + "operator": "ge", + "actual_fraction": 0.6144794432087249, + "metric": 0.8879242304656669, + "rescued": 2415, + "harmed": 691, + "missed_rescue": 376, + "avoided_harm": 199, + "selected_both_correct": 5048, + "selected_both_wrong": 410 + }, + { + "budget": 0.62, + "threshold": -0.9971383810043335, + "operator": "ge", + "actual_fraction": 0.6232331204706896, + "metric": 0.8888569993542369, + "rescued": 2435, + "harmed": 698, + "missed_rescue": 356, + "avoided_harm": 192, + "selected_both_correct": 5140, + "selected_both_wrong": 413 + }, + { + "budget": 0.63, + "threshold": -0.997377872467041, + "operator": "ge", + "actual_fraction": 0.6337805840568271, + "metric": 0.8894310109779723, + "rescued": 2451, + "harmed": 706, + "missed_rescue": 340, + "avoided_harm": 184, + "selected_both_correct": 5258, + "selected_both_wrong": 418 + }, + { + "budget": 0.64, + "threshold": -0.9976021647453308, + "operator": "ge", + "actual_fraction": 0.6445433020018655, + "metric": 0.8904355313195093, + "rescued": 2472, + "harmed": 713, + "missed_rescue": 319, + "avoided_harm": 177, + "selected_both_correct": 5375, + "selected_both_wrong": 423 + }, + { + "budget": 0.65, + "threshold": -0.9978214502334595, + "operator": "ge", + "actual_fraction": 0.6565257946473416, + "metric": 0.8904355313195093, + "rescued": 2488, + "harmed": 729, + "missed_rescue": 303, + "avoided_harm": 161, + "selected_both_correct": 5508, + "selected_both_wrong": 425 + }, + { + "budget": 0.66, + "threshold": -0.9980514049530029, + "operator": "ge", + "actual_fraction": 0.6692258018224869, + "metric": 0.8909377914902776, + "rescued": 2507, + "harmed": 741, + "missed_rescue": 284, + "avoided_harm": 149, + "selected_both_correct": 5650, + "selected_both_wrong": 429 + }, + { + "budget": 0.67, + "threshold": -0.9982016086578369, + "operator": "ge", + "actual_fraction": 0.6777642247255506, + "metric": 0.8917988089258807, + "rescued": 2523, + "harmed": 745, + "missed_rescue": 268, + "avoided_harm": 145, + "selected_both_correct": 5748, + "selected_both_wrong": 430 + }, + { + "budget": 0.68, + "threshold": -0.9983693957328796, + "operator": "ge", + "actual_fraction": 0.6885269426705891, + "metric": 0.8920858147377484, + "rescued": 2541, + "harmed": 759, + "missed_rescue": 250, + "avoided_harm": 131, + "selected_both_correct": 5864, + "selected_both_wrong": 432 + }, + { + "budget": 0.69, + "threshold": -0.9984915256500244, + "operator": "ge", + "actual_fraction": 0.6977828801033221, + "metric": 0.89251632345555, + "rescued": 2552, + "harmed": 764, + "missed_rescue": 239, + "avoided_harm": 126, + "selected_both_correct": 5973, + "selected_both_wrong": 436 + }, + { + "budget": 0.7, + "threshold": -0.9986109733581543, + "operator": "gt", + "actual_fraction": 0.7064648059123197, + "metric": 0.8928750807203846, + "rescued": 2563, + "harmed": 770, + "missed_rescue": 228, + "avoided_harm": 120, + "selected_both_correct": 6077, + "selected_both_wrong": 436 + }, + { + "budget": 0.71, + "threshold": -0.9986966252326965, + "operator": "ge", + "actual_fraction": 0.7137117026619789, + "metric": 0.8933055894381862, + "rescued": 2575, + "harmed": 776, + "missed_rescue": 216, + "avoided_harm": 114, + "selected_both_correct": 6154, + "selected_both_wrong": 442 + }, + { + "budget": 0.72, + "threshold": -0.9987963438034058, + "operator": "ge", + "actual_fraction": 0.7256941953074549, + "metric": 0.8941666068737892, + "rescued": 2592, + "harmed": 781, + "missed_rescue": 199, + "avoided_harm": 109, + "selected_both_correct": 6295, + "selected_both_wrong": 446 + }, + { + "budget": 0.73, + "threshold": -0.998875081539154, + "operator": "ge", + "actual_fraction": 0.7345196240223865, + "metric": 0.8950993757623592, + "rescued": 2605, + "harmed": 781, + "missed_rescue": 186, + "avoided_harm": 109, + "selected_both_correct": 6403, + "selected_both_wrong": 448 + }, + { + "budget": 0.74, + "threshold": -0.9989551305770874, + "operator": "ge", + "actual_fraction": 0.745067087608524, + "metric": 0.8956733873860946, + "rescued": 2618, + "harmed": 786, + "missed_rescue": 173, + "avoided_harm": 104, + "selected_both_correct": 6530, + "selected_both_wrong": 450 + }, + { + "budget": 0.75, + "threshold": -0.9990480542182922, + "operator": "ge", + "actual_fraction": 0.7568343258950994, + "metric": 0.8963191504627969, + "rescued": 2632, + "harmed": 791, + "missed_rescue": 159, + "avoided_harm": 99, + "selected_both_correct": 6671, + "selected_both_wrong": 454 + }, + { + "budget": 0.76, + "threshold": -0.999126136302948, + "operator": "ge", + "actual_fraction": 0.7663772691397001, + "metric": 0.8967496591805985, + "rescued": 2644, + "harmed": 797, + "missed_rescue": 147, + "avoided_harm": 93, + "selected_both_correct": 6783, + "selected_both_wrong": 457 + }, + { + "budget": 0.77, + "threshold": -0.999184787273407, + "operator": "gt", + "actual_fraction": 0.7746286862308962, + "metric": 0.8969649135394991, + "rescued": 2653, + "harmed": 803, + "missed_rescue": 138, + "avoided_harm": 87, + "selected_both_correct": 6883, + "selected_both_wrong": 457 + }, + { + "budget": 0.78, + "threshold": -0.9992268085479736, + "operator": "ge", + "actual_fraction": 0.7817320800746215, + "metric": 0.8966061562746646, + "rescued": 2657, + "harmed": 812, + "missed_rescue": 134, + "avoided_harm": 78, + "selected_both_correct": 6969, + "selected_both_wrong": 457 + }, + { + "budget": 0.79, + "threshold": -0.9992831349372864, + "operator": "ge", + "actual_fraction": 0.7928535552844945, + "metric": 0.8967496591805985, + "rescued": 2667, + "harmed": 820, + "missed_rescue": 124, + "avoided_harm": 70, + "selected_both_correct": 7106, + "selected_both_wrong": 457 + }, + { + "budget": 0.8, + "threshold": -0.9993318915367126, + "operator": "ge", + "actual_fraction": 0.8023247470761283, + "metric": 0.8966061562746646, + "rescued": 2669, + "harmed": 824, + "missed_rescue": 122, + "avoided_harm": 66, + "selected_both_correct": 7231, + "selected_both_wrong": 458 + }, + { + "budget": 0.81, + "threshold": -0.9993773102760315, + "operator": "ge", + "actual_fraction": 0.811795938867762, + "metric": 0.8968214106335654, + "rescued": 2677, + "harmed": 829, + "missed_rescue": 114, + "avoided_harm": 61, + "selected_both_correct": 7349, + "selected_both_wrong": 459 + }, + { + "budget": 0.82, + "threshold": -0.9994175434112549, + "operator": "ge", + "actual_fraction": 0.8214106335653297, + "metric": 0.8973954222573007, + "rescued": 2689, + "harmed": 833, + "missed_rescue": 102, + "avoided_harm": 57, + "selected_both_correct": 7465, + "selected_both_wrong": 461 + }, + { + "budget": 0.83, + "threshold": -0.999452531337738, + "operator": "ge", + "actual_fraction": 0.8298773050154266, + "metric": 0.8976824280691684, + "rescued": 2696, + "harmed": 836, + "missed_rescue": 95, + "avoided_harm": 54, + "selected_both_correct": 7572, + "selected_both_wrong": 462 + }, + { + "budget": 0.84, + "threshold": -0.9994945526123047, + "operator": "ge", + "actual_fraction": 0.8420750520198034, + "metric": 0.8973954222573007, + "rescued": 2702, + "harmed": 846, + "missed_rescue": 89, + "avoided_harm": 44, + "selected_both_correct": 7725, + "selected_both_wrong": 463 + }, + { + "budget": 0.85, + "threshold": -0.9995274543762207, + "operator": "gt", + "actual_fraction": 0.8502547176580326, + "metric": 0.8976106766162015, + "rescued": 2707, + "harmed": 848, + "missed_rescue": 84, + "avoided_harm": 42, + "selected_both_correct": 7829, + "selected_both_wrong": 466 + }, + { + "budget": 0.86, + "threshold": -0.9995617270469666, + "operator": "ge", + "actual_fraction": 0.8604434239793356, + "metric": 0.8976824280691684, + "rescued": 2712, + "harmed": 852, + "missed_rescue": 79, + "avoided_harm": 38, + "selected_both_correct": 7959, + "selected_both_wrong": 469 + }, + { + "budget": 0.87, + "threshold": -0.9995949864387512, + "operator": "ge", + "actual_fraction": 0.8690535983353663, + "metric": 0.8978259309751022, + "rescued": 2717, + "harmed": 855, + "missed_rescue": 74, + "avoided_harm": 35, + "selected_both_correct": 8069, + "selected_both_wrong": 471 + }, + { + "budget": 0.88, + "threshold": -0.9996248483657837, + "operator": "ge", + "actual_fraction": 0.8778790270502977, + "metric": 0.8978976824280692, + "rescued": 2723, + "harmed": 860, + "missed_rescue": 68, + "avoided_harm": 30, + "selected_both_correct": 8180, + "selected_both_wrong": 472 + }, + { + "budget": 0.89, + "threshold": -0.9996511936187744, + "operator": "ge", + "actual_fraction": 0.886058692688527, + "metric": 0.8981129367869699, + "rescued": 2731, + "harmed": 865, + "missed_rescue": 60, + "avoided_harm": 25, + "selected_both_correct": 8279, + "selected_both_wrong": 474 + }, + { + "budget": 0.9, + "threshold": -0.9996830224990845, + "operator": "ge", + "actual_fraction": 0.8961756475568631, + "metric": 0.8986869484107053, + "rescued": 2740, + "harmed": 866, + "missed_rescue": 51, + "avoided_harm": 24, + "selected_both_correct": 8407, + "selected_both_wrong": 477 + }, + { + "budget": 0.91, + "threshold": -0.9997097849845886, + "operator": "gt", + "actual_fraction": 0.9053598335366291, + "metric": 0.8990457056755399, + "rescued": 2746, + "harmed": 867, + "missed_rescue": 45, + "avoided_harm": 23, + "selected_both_correct": 8528, + "selected_both_wrong": 477 + }, + { + "budget": 0.92, + "threshold": -0.9997391104698181, + "operator": "ge", + "actual_fraction": 0.9164813087465021, + "metric": 0.8992609600344407, + "rescued": 2752, + "harmed": 870, + "missed_rescue": 39, + "avoided_harm": 20, + "selected_both_correct": 8672, + "selected_both_wrong": 479 + }, + { + "budget": 0.93, + "threshold": -0.9997641444206238, + "operator": "ge", + "actual_fraction": 0.9265265121618713, + "metric": 0.8995479658463084, + "rescued": 2759, + "harmed": 873, + "missed_rescue": 32, + "avoided_harm": 17, + "selected_both_correct": 8801, + "selected_both_wrong": 480 + }, + { + "budget": 0.94, + "threshold": -0.9997876286506653, + "operator": "ge", + "actual_fraction": 0.9362847097653727, + "metric": 0.8997632202052092, + "rescued": 2765, + "harmed": 876, + "missed_rescue": 26, + "avoided_harm": 14, + "selected_both_correct": 8926, + "selected_both_wrong": 482 + }, + { + "budget": 0.95, + "threshold": -0.9998131394386292, + "operator": "ge", + "actual_fraction": 0.9475496878811795, + "metric": 0.899906723111143, + "rescued": 2769, + "harmed": 878, + "missed_rescue": 22, + "avoided_harm": 12, + "selected_both_correct": 9075, + "selected_both_wrong": 484 + }, + { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9593169261677549, + "metric": 0.8996197172992753, + "rescued": 2770, + "harmed": 883, + "missed_rescue": 21, + "avoided_harm": 7, + "selected_both_correct": 9230, + "selected_both_wrong": 487 + }, + { + "budget": 0.97, + "threshold": -0.999868631362915, + "operator": "ge", + "actual_fraction": 0.9700796441127932, + "metric": 0.89997847456411, + "rescued": 2775, + "harmed": 883, + "missed_rescue": 16, + "avoided_harm": 7, + "selected_both_correct": 9371, + "selected_both_wrong": 491 + }, + { + "budget": 0.98, + "threshold": -0.9998925924301147, + "operator": "ge", + "actual_fraction": 0.9793355815455262, + "metric": 0.9001937289230106, + "rescued": 2780, + "harmed": 885, + "missed_rescue": 11, + "avoided_harm": 5, + "selected_both_correct": 9491, + "selected_both_wrong": 493 + }, + { + "budget": 0.99, + "threshold": -0.99992835521698, + "operator": "ge", + "actual_fraction": 0.9909593169261678, + "metric": 0.9004089832819114, + "rescued": 2786, + "harmed": 888, + "missed_rescue": 5, + "avoided_harm": 2, + "selected_both_correct": 9643, + "selected_both_wrong": 494 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9006242376408122, + "rescued": 2791, + "harmed": 890, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9761, + "selected_both_wrong": 495 + } + ] + }, + "learned": { + "selected_budget_index": 95, + "validation_area_0_25": 0.7963477513606416, + "validation_point": { + "budget": 0.95, + "threshold": -7.218595027923584, + "operator": "ge", + "actual_fraction": 0.9498710971068461, + "metric": 0.9041821827556574, + "rescued": 1431, + "harmed": 394, + "missed_rescue": 18, + "avoided_harm": 12, + "selected_both_correct": 4553, + "selected_both_wrong": 254 + }, + "test_point": { + "budget": 0.95, + "threshold": -7.218595027923584, + "operator": "ge", + "actual_fraction": 0.9485542082227165, + "metric": 0.89997847456411, + "rescued": 2767, + "harmed": 875, + "missed_rescue": 24, + "avoided_harm": 15, + "selected_both_correct": 9090, + "selected_both_wrong": 488 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7556574047550845, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1449, + "avoided_harm": 406, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.597435712814331, + "operator": "ge", + "actual_fraction": 0.009882555141793183, + "metric": 0.7598109424233744, + "rescued": 30, + "harmed": 1, + "missed_rescue": 1419, + "avoided_harm": 405, + "selected_both_correct": 33, + "selected_both_wrong": 5 + }, + { + "budget": 0.02, + "threshold": 1.389570713043213, + "operator": "ge", + "actual_fraction": 0.01990833572042395, + "metric": 0.7639644800916643, + "rescued": 59, + "harmed": 1, + "missed_rescue": 1390, + "avoided_harm": 405, + "selected_both_correct": 69, + "selected_both_wrong": 10 + }, + { + "budget": 0.03, + "threshold": 1.2446520328521729, + "operator": "ge", + "actual_fraction": 0.029934116299054712, + "metric": 0.7676883414494414, + "rescued": 91, + "harmed": 7, + "missed_rescue": 1358, + "avoided_harm": 399, + "selected_both_correct": 98, + "selected_both_wrong": 13 + }, + { + "budget": 0.04, + "threshold": 1.1426676511764526, + "operator": "ge", + "actual_fraction": 0.039959896877685475, + "metric": 0.7706960756230307, + "rescued": 116, + "harmed": 11, + "missed_rescue": 1333, + "avoided_harm": 395, + "selected_both_correct": 133, + "selected_both_wrong": 19 + }, + { + "budget": 0.05, + "threshold": 1.0597102642059326, + "operator": "ge", + "actual_fraction": 0.04998567745631624, + "metric": 0.7744199369808078, + "rescued": 145, + "harmed": 14, + "missed_rescue": 1304, + "avoided_harm": 392, + "selected_both_correct": 165, + "selected_both_wrong": 25 + }, + { + "budget": 0.06, + "threshold": 0.9505575299263, + "operator": "ge", + "actual_fraction": 0.059868232598109423, + "metric": 0.7784302492122601, + "rescued": 175, + "harmed": 16, + "missed_rescue": 1274, + "avoided_harm": 390, + "selected_both_correct": 192, + "selected_both_wrong": 35 + }, + { + "budget": 0.07, + "threshold": 0.8691579103469849, + "operator": "ge", + "actual_fraction": 0.06989401317674018, + "metric": 0.7815812088226869, + "rescued": 200, + "harmed": 19, + "missed_rescue": 1249, + "avoided_harm": 387, + "selected_both_correct": 231, + "selected_both_wrong": 38 + }, + { + "budget": 0.08, + "threshold": 0.7918534278869629, + "operator": "ge", + "actual_fraction": 0.07991979375537095, + "metric": 0.784302492122601, + "rescued": 221, + "harmed": 21, + "missed_rescue": 1228, + "avoided_harm": 385, + "selected_both_correct": 270, + "selected_both_wrong": 46 + }, + { + "budget": 0.09, + "threshold": 0.7071764469146729, + "operator": "ge", + "actual_fraction": 0.08994557433400172, + "metric": 0.7860211973646519, + "rescued": 241, + "harmed": 29, + "missed_rescue": 1208, + "avoided_harm": 377, + "selected_both_correct": 307, + "selected_both_wrong": 51 + }, + { + "budget": 0.1, + "threshold": 0.624683141708374, + "operator": "ge", + "actual_fraction": 0.09997135491263248, + "metric": 0.7893153824119163, + "rescued": 269, + "harmed": 34, + "missed_rescue": 1180, + "avoided_harm": 372, + "selected_both_correct": 342, + "selected_both_wrong": 53 + }, + { + "budget": 0.11, + "threshold": 0.5381495952606201, + "operator": "ge", + "actual_fraction": 0.10999713549126325, + "metric": 0.7917502148381552, + "rescued": 290, + "harmed": 38, + "missed_rescue": 1159, + "avoided_harm": 368, + "selected_both_correct": 387, + "selected_both_wrong": 53 + }, + { + "budget": 0.12, + "threshold": 0.4641508460044861, + "operator": "ge", + "actual_fraction": 0.11987969063305644, + "metric": 0.7951876253222573, + "rescued": 320, + "harmed": 44, + "missed_rescue": 1129, + "avoided_harm": 362, + "selected_both_correct": 420, + "selected_both_wrong": 53 + }, + { + "budget": 0.13, + "threshold": 0.39032384753227234, + "operator": "ge", + "actual_fraction": 0.1299054712116872, + "metric": 0.7983385849326841, + "rescued": 346, + "harmed": 48, + "missed_rescue": 1103, + "avoided_harm": 358, + "selected_both_correct": 455, + "selected_both_wrong": 58 + }, + { + "budget": 0.14, + "threshold": 0.3261690139770508, + "operator": "ge", + "actual_fraction": 0.13993125179031796, + "metric": 0.8022056717272987, + "rescued": 379, + "harmed": 54, + "missed_rescue": 1070, + "avoided_harm": 352, + "selected_both_correct": 480, + "selected_both_wrong": 64 + }, + { + "budget": 0.15, + "threshold": 0.26289427280426025, + "operator": "ge", + "actual_fraction": 0.14995703236894872, + "metric": 0.8065024348324262, + "rescued": 412, + "harmed": 57, + "missed_rescue": 1037, + "avoided_harm": 349, + "selected_both_correct": 509, + "selected_both_wrong": 69 + }, + { + "budget": 0.16, + "threshold": 0.16835230588912964, + "operator": "ge", + "actual_fraction": 0.1599828129475795, + "metric": 0.8097966198796906, + "rescued": 439, + "harmed": 61, + "missed_rescue": 1010, + "avoided_harm": 345, + "selected_both_correct": 546, + "selected_both_wrong": 71 + }, + { + "budget": 0.17, + "threshold": 0.09442993253469467, + "operator": "ge", + "actual_fraction": 0.16986536808937266, + "metric": 0.8129475794901174, + "rescued": 464, + "harmed": 64, + "missed_rescue": 985, + "avoided_harm": 342, + "selected_both_correct": 582, + "selected_both_wrong": 76 + }, + { + "budget": 0.18, + "threshold": 0.03206859529018402, + "operator": "ge", + "actual_fraction": 0.17989114866800343, + "metric": 0.8139501575479805, + "rescued": 484, + "harmed": 77, + "missed_rescue": 965, + "avoided_harm": 329, + "selected_both_correct": 616, + "selected_both_wrong": 79 + }, + { + "budget": 0.19, + "threshold": -0.0627312958240509, + "operator": "ge", + "actual_fraction": 0.1899169292466342, + "metric": 0.8158120882268691, + "rescued": 504, + "harmed": 84, + "missed_rescue": 945, + "avoided_harm": 322, + "selected_both_correct": 655, + "selected_both_wrong": 83 + }, + { + "budget": 0.2, + "threshold": -0.14934782683849335, + "operator": "ge", + "actual_fraction": 0.19994270982526496, + "metric": 0.818246920653108, + "rescued": 528, + "harmed": 91, + "missed_rescue": 921, + "avoided_harm": 315, + "selected_both_correct": 689, + "selected_both_wrong": 88 + }, + { + "budget": 0.21, + "threshold": -0.21385693550109863, + "operator": "ge", + "actual_fraction": 0.20996849040389573, + "metric": 0.8196791750214838, + "rescued": 550, + "harmed": 103, + "missed_rescue": 899, + "avoided_harm": 303, + "selected_both_correct": 720, + "selected_both_wrong": 93 + }, + { + "budget": 0.22, + "threshold": -0.2964950203895569, + "operator": "ge", + "actual_fraction": 0.2199942709825265, + "metric": 0.822686909195073, + "rescued": 574, + "harmed": 106, + "missed_rescue": 875, + "avoided_harm": 300, + "selected_both_correct": 760, + "selected_both_wrong": 96 + }, + { + "budget": 0.23, + "threshold": -0.37567442655563354, + "operator": "ge", + "actual_fraction": 0.22987682612431967, + "metric": 0.8255514179318247, + "rescued": 601, + "harmed": 113, + "missed_rescue": 848, + "avoided_harm": 293, + "selected_both_correct": 791, + "selected_both_wrong": 100 + }, + { + "budget": 0.24, + "threshold": -0.46664345264434814, + "operator": "ge", + "actual_fraction": 0.23990260670295044, + "metric": 0.8284159266685763, + "rescued": 623, + "harmed": 115, + "missed_rescue": 826, + "avoided_harm": 291, + "selected_both_correct": 834, + "selected_both_wrong": 103 + }, + { + "budget": 0.25, + "threshold": -0.5839477777481079, + "operator": "ge", + "actual_fraction": 0.2499283872815812, + "metric": 0.8308507590948152, + "rescued": 646, + "harmed": 121, + "missed_rescue": 803, + "avoided_harm": 285, + "selected_both_correct": 871, + "selected_both_wrong": 107 + }, + { + "budget": 0.26, + "threshold": -0.6750785708427429, + "operator": "ge", + "actual_fraction": 0.25995416786021197, + "metric": 0.8335720423947293, + "rescued": 670, + "harmed": 126, + "missed_rescue": 779, + "avoided_harm": 280, + "selected_both_correct": 911, + "selected_both_wrong": 108 + }, + { + "budget": 0.27, + "threshold": -0.7729678153991699, + "operator": "ge", + "actual_fraction": 0.26997994843884277, + "metric": 0.8354339730736179, + "rescued": 688, + "harmed": 131, + "missed_rescue": 761, + "avoided_harm": 275, + "selected_both_correct": 953, + "selected_both_wrong": 113 + }, + { + "budget": 0.28, + "threshold": -0.8474712371826172, + "operator": "ge", + "actual_fraction": 0.2798625035806359, + "metric": 0.8375823546261816, + "rescued": 708, + "harmed": 136, + "missed_rescue": 741, + "avoided_harm": 270, + "selected_both_correct": 991, + "selected_both_wrong": 119 + }, + { + "budget": 0.29, + "threshold": -0.947313666343689, + "operator": "ge", + "actual_fraction": 0.2898882841592667, + "metric": 0.8388713835577198, + "rescued": 723, + "harmed": 142, + "missed_rescue": 726, + "avoided_harm": 264, + "selected_both_correct": 1038, + "selected_both_wrong": 121 + }, + { + "budget": 0.3, + "threshold": -1.022983193397522, + "operator": "ge", + "actual_fraction": 0.29991406473789745, + "metric": 0.8415926668576339, + "rescued": 747, + "harmed": 147, + "missed_rescue": 702, + "avoided_harm": 259, + "selected_both_correct": 1072, + "selected_both_wrong": 128 + }, + { + "budget": 0.31, + "threshold": -1.1168640851974487, + "operator": "ge", + "actual_fraction": 0.30993984531652824, + "metric": 0.8435978229733601, + "rescued": 769, + "harmed": 155, + "missed_rescue": 680, + "avoided_harm": 251, + "selected_both_correct": 1108, + "selected_both_wrong": 132 + }, + { + "budget": 0.32, + "threshold": -1.2008401155471802, + "operator": "ge", + "actual_fraction": 0.319965625895159, + "metric": 0.8471784588942997, + "rescued": 796, + "harmed": 157, + "missed_rescue": 653, + "avoided_harm": 249, + "selected_both_correct": 1148, + "selected_both_wrong": 133 + }, + { + "budget": 0.33, + "threshold": -1.281516194343567, + "operator": "ge", + "actual_fraction": 0.3299914064737898, + "metric": 0.8493268404468634, + "rescued": 817, + "harmed": 163, + "missed_rescue": 632, + "avoided_harm": 243, + "selected_both_correct": 1185, + "selected_both_wrong": 139 + }, + { + "budget": 0.34, + "threshold": -1.360337734222412, + "operator": "ge", + "actual_fraction": 0.3398739616155829, + "metric": 0.851475221999427, + "rescued": 837, + "harmed": 168, + "missed_rescue": 612, + "avoided_harm": 238, + "selected_both_correct": 1224, + "selected_both_wrong": 144 + }, + { + "budget": 0.35, + "threshold": -1.444338321685791, + "operator": "ge", + "actual_fraction": 0.3498997421942137, + "metric": 0.8520481237467774, + "rescued": 850, + "harmed": 177, + "missed_rescue": 599, + "avoided_harm": 229, + "selected_both_correct": 1269, + "selected_both_wrong": 147 + }, + { + "budget": 0.36, + "threshold": -1.537778615951538, + "operator": "ge", + "actual_fraction": 0.35992552277284445, + "metric": 0.8534803781151532, + "rescued": 868, + "harmed": 185, + "missed_rescue": 581, + "avoided_harm": 221, + "selected_both_correct": 1308, + "selected_both_wrong": 152 + }, + { + "budget": 0.37, + "threshold": -1.6502166986465454, + "operator": "ge", + "actual_fraction": 0.36995130335147525, + "metric": 0.8551990833572043, + "rescued": 886, + "harmed": 191, + "missed_rescue": 563, + "avoided_harm": 215, + "selected_both_correct": 1349, + "selected_both_wrong": 157 + }, + { + "budget": 0.38, + "threshold": -1.748499870300293, + "operator": "ge", + "actual_fraction": 0.379977083930106, + "metric": 0.857347464909768, + "rescued": 907, + "harmed": 197, + "missed_rescue": 542, + "avoided_harm": 209, + "selected_both_correct": 1390, + "selected_both_wrong": 159 + }, + { + "budget": 0.39, + "threshold": -1.8659753799438477, + "operator": "ge", + "actual_fraction": 0.3898596390718992, + "metric": 0.8594958464623317, + "rescued": 929, + "harmed": 204, + "missed_rescue": 520, + "avoided_harm": 202, + "selected_both_correct": 1427, + "selected_both_wrong": 162 + }, + { + "budget": 0.4, + "threshold": -1.9553331136703491, + "operator": "ge", + "actual_fraction": 0.3998854196505299, + "metric": 0.8602119736465196, + "rescued": 942, + "harmed": 212, + "missed_rescue": 507, + "avoided_harm": 194, + "selected_both_correct": 1473, + "selected_both_wrong": 165 + }, + { + "budget": 0.41, + "threshold": -2.089120388031006, + "operator": "ge", + "actual_fraction": 0.4099112002291607, + "metric": 0.8613577771412203, + "rescued": 956, + "harmed": 218, + "missed_rescue": 493, + "avoided_harm": 188, + "selected_both_correct": 1521, + "selected_both_wrong": 167 + }, + { + "budget": 0.42, + "threshold": -2.2022628784179688, + "operator": "ge", + "actual_fraction": 0.41993698080779146, + "metric": 0.8623603551990834, + "rescued": 968, + "harmed": 223, + "missed_rescue": 481, + "avoided_harm": 183, + "selected_both_correct": 1570, + "selected_both_wrong": 171 + }, + { + "budget": 0.43, + "threshold": -2.310732841491699, + "operator": "ge", + "actual_fraction": 0.42996276138642225, + "metric": 0.8649384130621598, + "rescued": 987, + "harmed": 224, + "missed_rescue": 462, + "avoided_harm": 182, + "selected_both_correct": 1616, + "selected_both_wrong": 175 + }, + { + "budget": 0.44, + "threshold": -2.4193384647369385, + "operator": "ge", + "actual_fraction": 0.439988541965053, + "metric": 0.8663706674305357, + "rescued": 1001, + "harmed": 228, + "missed_rescue": 448, + "avoided_harm": 178, + "selected_both_correct": 1665, + "selected_both_wrong": 178 + }, + { + "budget": 0.45, + "threshold": -2.539083242416382, + "operator": "ge", + "actual_fraction": 0.4498710971068462, + "metric": 0.8685190489830994, + "rescued": 1018, + "harmed": 230, + "missed_rescue": 431, + "avoided_harm": 176, + "selected_both_correct": 1711, + "selected_both_wrong": 182 + }, + { + "budget": 0.46, + "threshold": -2.6633739471435547, + "operator": "ge", + "actual_fraction": 0.45989687768547693, + "metric": 0.8693784016041249, + "rescued": 1033, + "harmed": 239, + "missed_rescue": 416, + "avoided_harm": 167, + "selected_both_correct": 1756, + "selected_both_wrong": 183 + }, + { + "budget": 0.47, + "threshold": -2.7973411083221436, + "operator": "ge", + "actual_fraction": 0.46992265826410773, + "metric": 0.8700945287883128, + "rescued": 1044, + "harmed": 245, + "missed_rescue": 405, + "avoided_harm": 161, + "selected_both_correct": 1805, + "selected_both_wrong": 187 + }, + { + "budget": 0.48, + "threshold": -2.923297166824341, + "operator": "ge", + "actual_fraction": 0.47994843884273847, + "metric": 0.872099684904039, + "rescued": 1062, + "harmed": 249, + "missed_rescue": 387, + "avoided_harm": 157, + "selected_both_correct": 1849, + "selected_both_wrong": 191 + }, + { + "budget": 0.49, + "threshold": -3.0752992630004883, + "operator": "ge", + "actual_fraction": 0.48997421942136926, + "metric": 0.8732454883987396, + "rescued": 1076, + "harmed": 255, + "missed_rescue": 373, + "avoided_harm": 151, + "selected_both_correct": 1897, + "selected_both_wrong": 193 + }, + { + "budget": 0.5, + "threshold": -3.220832347869873, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.8749641936407906, + "rescued": 1091, + "harmed": 258, + "missed_rescue": 358, + "avoided_harm": 148, + "selected_both_correct": 1944, + "selected_both_wrong": 198 + }, + { + "budget": 0.51, + "threshold": -3.3313610553741455, + "operator": "ge", + "actual_fraction": 0.5098825551417931, + "metric": 0.8756803208249785, + "rescued": 1102, + "harmed": 264, + "missed_rescue": 347, + "avoided_harm": 142, + "selected_both_correct": 1993, + "selected_both_wrong": 201 + }, + { + "budget": 0.52, + "threshold": -3.453277587890625, + "operator": "ge", + "actual_fraction": 0.5199083357204239, + "metric": 0.8771125751933544, + "rescued": 1116, + "harmed": 268, + "missed_rescue": 333, + "avoided_harm": 138, + "selected_both_correct": 2044, + "selected_both_wrong": 202 + }, + { + "budget": 0.53, + "threshold": -3.5726635456085205, + "operator": "ge", + "actual_fraction": 0.5299341162990547, + "metric": 0.8773990260670295, + "rescued": 1121, + "harmed": 271, + "missed_rescue": 328, + "avoided_harm": 135, + "selected_both_correct": 2104, + "selected_both_wrong": 204 + }, + { + "budget": 0.54, + "threshold": -3.656648635864258, + "operator": "ge", + "actual_fraction": 0.5399598968776855, + "metric": 0.8789745058722429, + "rescued": 1133, + "harmed": 272, + "missed_rescue": 316, + "avoided_harm": 134, + "selected_both_correct": 2158, + "selected_both_wrong": 207 + }, + { + "budget": 0.55, + "threshold": -3.7778828144073486, + "operator": "ge", + "actual_fraction": 0.5499856774563162, + "metric": 0.8788312804354054, + "rescued": 1141, + "harmed": 281, + "missed_rescue": 308, + "avoided_harm": 125, + "selected_both_correct": 2211, + "selected_both_wrong": 207 + }, + { + "budget": 0.56, + "threshold": -3.869736671447754, + "operator": "ge", + "actual_fraction": 0.5598682325981095, + "metric": 0.8795474076195933, + "rescued": 1149, + "harmed": 284, + "missed_rescue": 300, + "avoided_harm": 122, + "selected_both_correct": 2266, + "selected_both_wrong": 210 + }, + { + "budget": 0.57, + "threshold": -3.9598283767700195, + "operator": "ge", + "actual_fraction": 0.5698940131767402, + "metric": 0.8802635348037812, + "rescued": 1158, + "harmed": 288, + "missed_rescue": 291, + "avoided_harm": 118, + "selected_both_correct": 2321, + "selected_both_wrong": 212 + }, + { + "budget": 0.58, + "threshold": -4.036509990692139, + "operator": "ge", + "actual_fraction": 0.579919793755371, + "metric": 0.8805499856774563, + "rescued": 1168, + "harmed": 296, + "missed_rescue": 281, + "avoided_harm": 110, + "selected_both_correct": 2372, + "selected_both_wrong": 213 + }, + { + "budget": 0.59, + "threshold": -4.113898754119873, + "operator": "ge", + "actual_fraction": 0.5899455743340017, + "metric": 0.8812661128616442, + "rescued": 1178, + "harmed": 301, + "missed_rescue": 271, + "avoided_harm": 105, + "selected_both_correct": 2426, + "selected_both_wrong": 214 + }, + { + "budget": 0.6, + "threshold": -4.190845966339111, + "operator": "ge", + "actual_fraction": 0.5999713549126325, + "metric": 0.8818390146089946, + "rescued": 1186, + "harmed": 305, + "missed_rescue": 263, + "avoided_harm": 101, + "selected_both_correct": 2484, + "selected_both_wrong": 214 + }, + { + "budget": 0.61, + "threshold": -4.2746686935424805, + "operator": "ge", + "actual_fraction": 0.6099971354912632, + "metric": 0.8828415926668577, + "rescued": 1196, + "harmed": 308, + "missed_rescue": 253, + "avoided_harm": 98, + "selected_both_correct": 2541, + "selected_both_wrong": 214 + }, + { + "budget": 0.62, + "threshold": -4.353496074676514, + "operator": "ge", + "actual_fraction": 0.6198796906330565, + "metric": 0.8837009452878831, + "rescued": 1205, + "harmed": 311, + "missed_rescue": 244, + "avoided_harm": 95, + "selected_both_correct": 2597, + "selected_both_wrong": 215 + }, + { + "budget": 0.63, + "threshold": -4.4249958992004395, + "operator": "ge", + "actual_fraction": 0.6299054712116872, + "metric": 0.8854196505299341, + "rescued": 1218, + "harmed": 312, + "missed_rescue": 231, + "avoided_harm": 94, + "selected_both_correct": 2650, + "selected_both_wrong": 218 + }, + { + "budget": 0.64, + "threshold": -4.508240699768066, + "operator": "ge", + "actual_fraction": 0.639931251790318, + "metric": 0.8858493268404468, + "rescued": 1225, + "harmed": 316, + "missed_rescue": 224, + "avoided_harm": 90, + "selected_both_correct": 2709, + "selected_both_wrong": 218 + }, + { + "budget": 0.65, + "threshold": -4.580889701843262, + "operator": "ge", + "actual_fraction": 0.6499570323689488, + "metric": 0.8865654540246348, + "rescued": 1234, + "harmed": 320, + "missed_rescue": 215, + "avoided_harm": 86, + "selected_both_correct": 2764, + "selected_both_wrong": 220 + }, + { + "budget": 0.66, + "threshold": -4.649904727935791, + "operator": "ge", + "actual_fraction": 0.6599828129475795, + "metric": 0.8869951303351475, + "rescued": 1239, + "harmed": 322, + "missed_rescue": 210, + "avoided_harm": 84, + "selected_both_correct": 2825, + "selected_both_wrong": 222 + }, + { + "budget": 0.67, + "threshold": -4.736553192138672, + "operator": "ge", + "actual_fraction": 0.6698653680893727, + "metric": 0.8882841592666858, + "rescued": 1250, + "harmed": 324, + "missed_rescue": 199, + "avoided_harm": 82, + "selected_both_correct": 2879, + "selected_both_wrong": 224 + }, + { + "budget": 0.68, + "threshold": -4.816196918487549, + "operator": "ge", + "actual_fraction": 0.6798911486680035, + "metric": 0.8894299627613864, + "rescued": 1259, + "harmed": 325, + "missed_rescue": 190, + "avoided_harm": 81, + "selected_both_correct": 2939, + "selected_both_wrong": 224 + }, + { + "budget": 0.69, + "threshold": -4.892243385314941, + "operator": "ge", + "actual_fraction": 0.6899169292466342, + "metric": 0.8900028645087368, + "rescued": 1266, + "harmed": 328, + "missed_rescue": 183, + "avoided_harm": 78, + "selected_both_correct": 2998, + "selected_both_wrong": 225 + }, + { + "budget": 0.7, + "threshold": -4.959829807281494, + "operator": "ge", + "actual_fraction": 0.699942709825265, + "metric": 0.890575766256087, + "rescued": 1272, + "harmed": 330, + "missed_rescue": 177, + "avoided_harm": 76, + "selected_both_correct": 3060, + "selected_both_wrong": 225 + }, + { + "budget": 0.71, + "threshold": -5.037147045135498, + "operator": "ge", + "actual_fraction": 0.7099684904038958, + "metric": 0.8917215697507878, + "rescued": 1285, + "harmed": 335, + "missed_rescue": 164, + "avoided_harm": 71, + "selected_both_correct": 3111, + "selected_both_wrong": 226 + }, + { + "budget": 0.72, + "threshold": -5.102000713348389, + "operator": "ge", + "actual_fraction": 0.7199942709825266, + "metric": 0.8925809223718132, + "rescued": 1293, + "harmed": 337, + "missed_rescue": 156, + "avoided_harm": 69, + "selected_both_correct": 3169, + "selected_both_wrong": 228 + }, + { + "budget": 0.73, + "threshold": -5.184895038604736, + "operator": "ge", + "actual_fraction": 0.7298768261243197, + "metric": 0.8925809223718132, + "rescued": 1297, + "harmed": 341, + "missed_rescue": 152, + "avoided_harm": 65, + "selected_both_correct": 3230, + "selected_both_wrong": 228 + }, + { + "budget": 0.74, + "threshold": -5.24793815612793, + "operator": "ge", + "actual_fraction": 0.7399026067029505, + "metric": 0.8935835004296763, + "rescued": 1307, + "harmed": 344, + "missed_rescue": 142, + "avoided_harm": 62, + "selected_both_correct": 3286, + "selected_both_wrong": 229 + }, + { + "budget": 0.75, + "threshold": -5.323313236236572, + "operator": "ge", + "actual_fraction": 0.7499283872815812, + "metric": 0.8932970495560012, + "rescued": 1311, + "harmed": 350, + "missed_rescue": 138, + "avoided_harm": 56, + "selected_both_correct": 3346, + "selected_both_wrong": 229 + }, + { + "budget": 0.76, + "threshold": -5.399226665496826, + "operator": "ge", + "actual_fraction": 0.759954167860212, + "metric": 0.8935835004296763, + "rescued": 1317, + "harmed": 354, + "missed_rescue": 132, + "avoided_harm": 52, + "selected_both_correct": 3405, + "selected_both_wrong": 230 + }, + { + "budget": 0.77, + "threshold": -5.4738969802856445, + "operator": "ge", + "actual_fraction": 0.7699799484388428, + "metric": 0.8948725293612145, + "rescued": 1327, + "harmed": 355, + "missed_rescue": 122, + "avoided_harm": 51, + "selected_both_correct": 3463, + "selected_both_wrong": 231 + }, + { + "budget": 0.78, + "threshold": -5.566734313964844, + "operator": "ge", + "actual_fraction": 0.7798625035806359, + "metric": 0.8954454311085649, + "rescued": 1334, + "harmed": 358, + "missed_rescue": 115, + "avoided_harm": 48, + "selected_both_correct": 3522, + "selected_both_wrong": 231 + }, + { + "budget": 0.79, + "threshold": -5.644944190979004, + "operator": "ge", + "actual_fraction": 0.7898882841592667, + "metric": 0.8954454311085649, + "rescued": 1336, + "harmed": 360, + "missed_rescue": 113, + "avoided_harm": 46, + "selected_both_correct": 3587, + "selected_both_wrong": 232 + }, + { + "budget": 0.8, + "threshold": -5.722663402557373, + "operator": "ge", + "actual_fraction": 0.7999140647378975, + "metric": 0.8958751074190776, + "rescued": 1342, + "harmed": 363, + "missed_rescue": 107, + "avoided_harm": 43, + "selected_both_correct": 3646, + "selected_both_wrong": 234 + }, + { + "budget": 0.81, + "threshold": -5.805805683135986, + "operator": "ge", + "actual_fraction": 0.8099398453165282, + "metric": 0.896448009166428, + "rescued": 1349, + "harmed": 366, + "missed_rescue": 100, + "avoided_harm": 40, + "selected_both_correct": 3703, + "selected_both_wrong": 237 + }, + { + "budget": 0.82, + "threshold": -5.893346786499023, + "operator": "ge", + "actual_fraction": 0.819965625895159, + "metric": 0.8973073617874534, + "rescued": 1356, + "harmed": 367, + "missed_rescue": 93, + "avoided_harm": 39, + "selected_both_correct": 3765, + "selected_both_wrong": 237 + }, + { + "budget": 0.83, + "threshold": -5.973089694976807, + "operator": "ge", + "actual_fraction": 0.8299914064737898, + "metric": 0.897450587224291, + "rescued": 1363, + "harmed": 373, + "missed_rescue": 86, + "avoided_harm": 33, + "selected_both_correct": 3822, + "selected_both_wrong": 237 + }, + { + "budget": 0.84, + "threshold": -6.067188739776611, + "operator": "ge", + "actual_fraction": 0.8398739616155829, + "metric": 0.8983099398453165, + "rescued": 1370, + "harmed": 374, + "missed_rescue": 79, + "avoided_harm": 32, + "selected_both_correct": 3882, + "selected_both_wrong": 238 + }, + { + "budget": 0.85, + "threshold": -6.1676530838012695, + "operator": "ge", + "actual_fraction": 0.8498997421942137, + "metric": 0.8987396161558293, + "rescued": 1376, + "harmed": 377, + "missed_rescue": 73, + "avoided_harm": 29, + "selected_both_correct": 3941, + "selected_both_wrong": 240 + }, + { + "budget": 0.86, + "threshold": -6.258213520050049, + "operator": "ge", + "actual_fraction": 0.8599255227728445, + "metric": 0.8990260670295045, + "rescued": 1379, + "harmed": 378, + "missed_rescue": 70, + "avoided_harm": 28, + "selected_both_correct": 4005, + "selected_both_wrong": 242 + }, + { + "budget": 0.87, + "threshold": -6.353724002838135, + "operator": "ge", + "actual_fraction": 0.8699513033514752, + "metric": 0.8993125179031796, + "rescued": 1384, + "harmed": 381, + "missed_rescue": 65, + "avoided_harm": 25, + "selected_both_correct": 4064, + "selected_both_wrong": 245 + }, + { + "budget": 0.88, + "threshold": -6.4612135887146, + "operator": "ge", + "actual_fraction": 0.879977083930106, + "metric": 0.9001718705242051, + "rescued": 1392, + "harmed": 383, + "missed_rescue": 57, + "avoided_harm": 23, + "selected_both_correct": 4124, + "selected_both_wrong": 245 + }, + { + "budget": 0.89, + "threshold": -6.545001983642578, + "operator": "ge", + "actual_fraction": 0.8898596390718991, + "metric": 0.9007447722715555, + "rescued": 1399, + "harmed": 386, + "missed_rescue": 50, + "avoided_harm": 20, + "selected_both_correct": 4181, + "selected_both_wrong": 247 + }, + { + "budget": 0.9, + "threshold": -6.64798641204834, + "operator": "ge", + "actual_fraction": 0.8998854196505299, + "metric": 0.9018905757662561, + "rescued": 1408, + "harmed": 387, + "missed_rescue": 41, + "avoided_harm": 19, + "selected_both_correct": 4240, + "selected_both_wrong": 248 + }, + { + "budget": 0.91, + "threshold": -6.747555255889893, + "operator": "ge", + "actual_fraction": 0.9099112002291607, + "metric": 0.9020338012030936, + "rescued": 1411, + "harmed": 389, + "missed_rescue": 38, + "avoided_harm": 17, + "selected_both_correct": 4303, + "selected_both_wrong": 250 + }, + { + "budget": 0.92, + "threshold": -6.860367774963379, + "operator": "ge", + "actual_fraction": 0.9199369808077915, + "metric": 0.9023202520767688, + "rescued": 1414, + "harmed": 390, + "missed_rescue": 35, + "avoided_harm": 16, + "selected_both_correct": 4365, + "selected_both_wrong": 254 + }, + { + "budget": 0.93, + "threshold": -6.962065696716309, + "operator": "ge", + "actual_fraction": 0.9299627613864222, + "metric": 0.9036092810083071, + "rescued": 1423, + "harmed": 390, + "missed_rescue": 26, + "avoided_harm": 16, + "selected_both_correct": 4426, + "selected_both_wrong": 254 + }, + { + "budget": 0.94, + "threshold": -7.099829196929932, + "operator": "ge", + "actual_fraction": 0.939988541965053, + "metric": 0.9036092810083071, + "rescued": 1426, + "harmed": 393, + "missed_rescue": 23, + "avoided_harm": 13, + "selected_both_correct": 4490, + "selected_both_wrong": 254 + }, + { + "budget": 0.95, + "threshold": -7.218595027923584, + "operator": "ge", + "actual_fraction": 0.9498710971068461, + "metric": 0.9041821827556574, + "rescued": 1431, + "harmed": 394, + "missed_rescue": 18, + "avoided_harm": 12, + "selected_both_correct": 4553, + "selected_both_wrong": 254 + }, + { + "budget": 0.96, + "threshold": -7.371253967285156, + "operator": "ge", + "actual_fraction": 0.9598968776854769, + "metric": 0.9037525064451447, + "rescued": 1433, + "harmed": 399, + "missed_rescue": 16, + "avoided_harm": 7, + "selected_both_correct": 4616, + "selected_both_wrong": 254 + }, + { + "budget": 0.97, + "threshold": -7.56823205947876, + "operator": "ge", + "actual_fraction": 0.9699226582641077, + "metric": 0.9038957318819822, + "rescued": 1434, + "harmed": 399, + "missed_rescue": 15, + "avoided_harm": 7, + "selected_both_correct": 4684, + "selected_both_wrong": 255 + }, + { + "budget": 0.98, + "threshold": -7.766812801361084, + "operator": "ge", + "actual_fraction": 0.9799484388427385, + "metric": 0.9040389573188198, + "rescued": 1438, + "harmed": 402, + "missed_rescue": 11, + "avoided_harm": 4, + "selected_both_correct": 4747, + "selected_both_wrong": 255 + }, + { + "budget": 0.99, + "threshold": -8.102746963500977, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.9041821827556574, + "rescued": 1441, + "harmed": 404, + "missed_rescue": 8, + "avoided_harm": 2, + "selected_both_correct": 4810, + "selected_both_wrong": 257 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9050415353766829, + "rescued": 1449, + "harmed": 406, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 4870, + "selected_both_wrong": 257 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7642247255506924, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2791, + "avoided_harm": 890, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.597435712814331, + "operator": "ge", + "actual_fraction": 0.013417521704814523, + "metric": 0.7681710554638731, + "rescued": 62, + "harmed": 7, + "missed_rescue": 2729, + "avoided_harm": 883, + "selected_both_correct": 101, + "selected_both_wrong": 17 + }, + { + "budget": 0.02, + "threshold": 1.389570713043213, + "operator": "ge", + "actual_fraction": 0.02504125708545598, + "metric": 0.7726196455478224, + "rescued": 131, + "harmed": 14, + "missed_rescue": 2660, + "avoided_harm": 876, + "selected_both_correct": 183, + "selected_both_wrong": 21 + }, + { + "budget": 0.03, + "threshold": 1.2446520328521729, + "operator": "ge", + "actual_fraction": 0.034010188706321304, + "metric": 0.7754179522135323, + "rescued": 177, + "harmed": 21, + "missed_rescue": 2614, + "avoided_harm": 869, + "selected_both_correct": 246, + "selected_both_wrong": 30 + }, + { + "budget": 0.04, + "threshold": 1.1426676511764526, + "operator": "ge", + "actual_fraction": 0.04197459998564971, + "metric": 0.7781445074262754, + "rescued": 221, + "harmed": 27, + "missed_rescue": 2570, + "avoided_harm": 863, + "selected_both_correct": 301, + "selected_both_wrong": 36 + }, + { + "budget": 0.05, + "threshold": 1.0597102642059326, + "operator": "ge", + "actual_fraction": 0.05001076271794504, + "metric": 0.7811580684508861, + "rescued": 265, + "harmed": 29, + "missed_rescue": 2526, + "avoided_harm": 861, + "selected_both_correct": 363, + "selected_both_wrong": 40 + }, + { + "budget": 0.06, + "threshold": 0.9505575299263, + "operator": "ge", + "actual_fraction": 0.06041472339814881, + "metric": 0.7846021381932984, + "rescued": 321, + "harmed": 37, + "missed_rescue": 2470, + "avoided_harm": 853, + "selected_both_correct": 434, + "selected_both_wrong": 50 + }, + { + "budget": 0.07, + "threshold": 0.8691579103469849, + "operator": "ge", + "actual_fraction": 0.06945540647198106, + "metric": 0.7876156992179092, + "rescued": 372, + "harmed": 46, + "missed_rescue": 2419, + "avoided_harm": 844, + "selected_both_correct": 491, + "selected_both_wrong": 59 + }, + { + "budget": 0.08, + "threshold": 0.7918534278869629, + "operator": "ge", + "actual_fraction": 0.07899834971658176, + "metric": 0.7913467747721892, + "rescued": 429, + "harmed": 51, + "missed_rescue": 2362, + "avoided_harm": 839, + "selected_both_correct": 553, + "selected_both_wrong": 68 + }, + { + "budget": 0.09, + "threshold": 0.7071764469146729, + "operator": "ge", + "actual_fraction": 0.08997632202052092, + "metric": 0.7952931046853698, + "rescued": 490, + "harmed": 57, + "missed_rescue": 2301, + "avoided_harm": 833, + "selected_both_correct": 629, + "selected_both_wrong": 78 + }, + { + "budget": 0.1, + "threshold": 0.624683141708374, + "operator": "ge", + "actual_fraction": 0.09973451962402238, + "metric": 0.7977326540862453, + "rescued": 532, + "harmed": 65, + "missed_rescue": 2259, + "avoided_harm": 825, + "selected_both_correct": 699, + "selected_both_wrong": 94 + }, + { + "budget": 0.11, + "threshold": 0.5381495952606201, + "operator": "ge", + "actual_fraction": 0.10970797158642462, + "metric": 0.8008179665638229, + "rescued": 588, + "harmed": 78, + "missed_rescue": 2203, + "avoided_harm": 812, + "selected_both_correct": 759, + "selected_both_wrong": 104 + }, + { + "budget": 0.12, + "threshold": 0.4641508460044861, + "operator": "ge", + "actual_fraction": 0.11810289158355457, + "metric": 0.8039032790414006, + "rescued": 638, + "harmed": 85, + "missed_rescue": 2153, + "avoided_harm": 805, + "selected_both_correct": 809, + "selected_both_wrong": 114 + }, + { + "budget": 0.13, + "threshold": 0.39032384753227234, + "operator": "ge", + "actual_fraction": 0.12750233192222143, + "metric": 0.8064863313482098, + "rescued": 682, + "harmed": 93, + "missed_rescue": 2109, + "avoided_harm": 797, + "selected_both_correct": 883, + "selected_both_wrong": 119 + }, + { + "budget": 0.14, + "threshold": 0.3261690139770508, + "operator": "ge", + "actual_fraction": 0.13467747721891368, + "metric": 0.8079931118605151, + "rescued": 718, + "harmed": 108, + "missed_rescue": 2073, + "avoided_harm": 782, + "selected_both_correct": 923, + "selected_both_wrong": 128 + }, + { + "budget": 0.15, + "threshold": 0.26289427280426025, + "operator": "ge", + "actual_fraction": 0.14213962832747362, + "metric": 0.8099304010906221, + "rescued": 752, + "harmed": 115, + "missed_rescue": 2039, + "avoided_harm": 775, + "selected_both_correct": 982, + "selected_both_wrong": 132 + }, + { + "budget": 0.16, + "threshold": 0.16835230588912964, + "operator": "ge", + "actual_fraction": 0.1534763578962474, + "metric": 0.8129439621152328, + "rescued": 805, + "harmed": 126, + "missed_rescue": 1986, + "avoided_harm": 764, + "selected_both_correct": 1067, + "selected_both_wrong": 141 + }, + { + "budget": 0.17, + "threshold": 0.09442993253469467, + "operator": "ge", + "actual_fraction": 0.1623017866111789, + "metric": 0.8153835115161082, + "rescued": 851, + "harmed": 138, + "missed_rescue": 1940, + "avoided_harm": 752, + "selected_both_correct": 1125, + "selected_both_wrong": 148 + }, + { + "budget": 0.18, + "threshold": 0.03206859529018402, + "operator": "ge", + "actual_fraction": 0.17005094353160652, + "metric": 0.8176078065580829, + "rescued": 890, + "harmed": 146, + "missed_rescue": 1901, + "avoided_harm": 744, + "selected_both_correct": 1178, + "selected_both_wrong": 156 + }, + { + "budget": 0.19, + "threshold": -0.0627312958240509, + "operator": "ge", + "actual_fraction": 0.18088541292961183, + "metric": 0.8209801248475281, + "rescued": 946, + "harmed": 155, + "missed_rescue": 1845, + "avoided_harm": 735, + "selected_both_correct": 1253, + "selected_both_wrong": 167 + }, + { + "budget": 0.2, + "threshold": -0.14934782683849335, + "operator": "ge", + "actual_fraction": 0.19207863959245175, + "metric": 0.8237784315132382, + "rescued": 996, + "harmed": 166, + "missed_rescue": 1795, + "avoided_harm": 724, + "selected_both_correct": 1339, + "selected_both_wrong": 176 + }, + { + "budget": 0.21, + "threshold": -0.21385693550109863, + "operator": "ge", + "actual_fraction": 0.20068881394848245, + "metric": 0.8264332352730143, + "rescued": 1038, + "harmed": 171, + "missed_rescue": 1753, + "avoided_harm": 719, + "selected_both_correct": 1408, + "selected_both_wrong": 180 + }, + { + "budget": 0.22, + "threshold": -0.2964950203895569, + "operator": "ge", + "actual_fraction": 0.2110210231757193, + "metric": 0.8282987730501543, + "rescued": 1081, + "harmed": 188, + "missed_rescue": 1710, + "avoided_harm": 702, + "selected_both_correct": 1478, + "selected_both_wrong": 194 + }, + { + "budget": 0.23, + "threshold": -0.37567442655563354, + "operator": "ge", + "actual_fraction": 0.21869842864318004, + "metric": 0.8306665709980627, + "rescued": 1121, + "harmed": 195, + "missed_rescue": 1670, + "avoided_harm": 695, + "selected_both_correct": 1529, + "selected_both_wrong": 203 + }, + { + "budget": 0.24, + "threshold": -0.46664345264434814, + "operator": "ge", + "actual_fraction": 0.22974815240008611, + "metric": 0.8329626174930043, + "rescued": 1169, + "harmed": 211, + "missed_rescue": 1622, + "avoided_harm": 679, + "selected_both_correct": 1606, + "selected_both_wrong": 216 + }, + { + "budget": 0.25, + "threshold": -0.5839477777481079, + "operator": "ge", + "actual_fraction": 0.24417019444643753, + "metric": 0.8364784386883835, + "rescued": 1236, + "harmed": 229, + "missed_rescue": 1555, + "avoided_harm": 661, + "selected_both_correct": 1715, + "selected_both_wrong": 223 + }, + { + "budget": 0.26, + "threshold": -0.6750785708427429, + "operator": "ge", + "actual_fraction": 0.25565042692114515, + "metric": 0.8386309822773911, + "rescued": 1286, + "harmed": 249, + "missed_rescue": 1505, + "avoided_harm": 641, + "selected_both_correct": 1798, + "selected_both_wrong": 230 + }, + { + "budget": 0.27, + "threshold": -0.7729678153991699, + "operator": "ge", + "actual_fraction": 0.2656238788835474, + "metric": 0.8409987802252996, + "rescued": 1331, + "harmed": 261, + "missed_rescue": 1460, + "avoided_harm": 629, + "selected_both_correct": 1870, + "selected_both_wrong": 240 + }, + { + "budget": 0.28, + "threshold": -0.8474712371826172, + "operator": "ge", + "actual_fraction": 0.27466456195737965, + "metric": 0.8425773121905719, + "rescued": 1364, + "harmed": 272, + "missed_rescue": 1427, + "avoided_harm": 618, + "selected_both_correct": 1947, + "selected_both_wrong": 245 + }, + { + "budget": 0.29, + "threshold": -0.947313666343689, + "operator": "ge", + "actual_fraction": 0.28765157494439264, + "metric": 0.8448016072325465, + "rescued": 1415, + "harmed": 292, + "missed_rescue": 1376, + "avoided_harm": 598, + "selected_both_correct": 2048, + "selected_both_wrong": 254 + }, + { + "budget": 0.3, + "threshold": -1.022983193397522, + "operator": "ge", + "actual_fraction": 0.2968357609241587, + "metric": 0.8463801391978187, + "rescued": 1451, + "harmed": 306, + "missed_rescue": 1340, + "avoided_harm": 584, + "selected_both_correct": 2121, + "selected_both_wrong": 259 + }, + { + "budget": 0.31, + "threshold": -1.1168640851974487, + "operator": "ge", + "actual_fraction": 0.30831599339886634, + "metric": 0.8486761856927603, + "rescued": 1496, + "harmed": 319, + "missed_rescue": 1295, + "avoided_harm": 571, + "selected_both_correct": 2217, + "selected_both_wrong": 265 + }, + { + "budget": 0.32, + "threshold": -1.2008401155471802, + "operator": "ge", + "actual_fraction": 0.3180024395494009, + "metric": 0.8498959603931979, + "rescued": 1531, + "harmed": 337, + "missed_rescue": 1260, + "avoided_harm": 553, + "selected_both_correct": 2294, + "selected_both_wrong": 270 + }, + { + "budget": 0.33, + "threshold": -1.281516194343567, + "operator": "ge", + "actual_fraction": 0.3267561168113654, + "metric": 0.8519767525292388, + "rescued": 1567, + "harmed": 344, + "missed_rescue": 1224, + "avoided_harm": 546, + "selected_both_correct": 2371, + "selected_both_wrong": 272 + }, + { + "budget": 0.34, + "threshold": -1.360337734222412, + "operator": "ge", + "actual_fraction": 0.3357250484322308, + "metric": 0.8536270359474779, + "rescued": 1601, + "harmed": 355, + "missed_rescue": 1190, + "avoided_harm": 535, + "selected_both_correct": 2449, + "selected_both_wrong": 274 + }, + { + "budget": 0.35, + "threshold": -1.444338321685791, + "operator": "ge", + "actual_fraction": 0.3446939800530961, + "metric": 0.8550620650068164, + "rescued": 1633, + "harmed": 367, + "missed_rescue": 1158, + "avoided_harm": 523, + "selected_both_correct": 2522, + "selected_both_wrong": 282 + }, + { + "budget": 0.36, + "threshold": -1.537778615951538, + "operator": "ge", + "actual_fraction": 0.35380641457989526, + "metric": 0.8567123484250556, + "rescued": 1662, + "harmed": 373, + "missed_rescue": 1129, + "avoided_harm": 517, + "selected_both_correct": 2603, + "selected_both_wrong": 293 + }, + { + "budget": 0.37, + "threshold": -1.6502166986465454, + "operator": "ge", + "actual_fraction": 0.3663629188491067, + "metric": 0.8595106550907656, + "rescued": 1714, + "harmed": 386, + "missed_rescue": 1077, + "avoided_harm": 504, + "selected_both_correct": 2705, + "selected_both_wrong": 301 + }, + { + "budget": 0.38, + "threshold": -1.748499870300293, + "operator": "ge", + "actual_fraction": 0.3751883475640382, + "metric": 0.8616631986797733, + "rescued": 1755, + "harmed": 397, + "missed_rescue": 1036, + "avoided_harm": 493, + "selected_both_correct": 2770, + "selected_both_wrong": 307 + }, + { + "budget": 0.39, + "threshold": -1.8659753799438477, + "operator": "ge", + "actual_fraction": 0.3865250771328119, + "metric": 0.8630982277391117, + "rescued": 1790, + "harmed": 412, + "missed_rescue": 1001, + "avoided_harm": 478, + "selected_both_correct": 2874, + "selected_both_wrong": 311 + }, + { + "budget": 0.4, + "threshold": -1.9553331136703491, + "operator": "ge", + "actual_fraction": 0.39484824567697496, + "metric": 0.8650355169692187, + "rescued": 1826, + "harmed": 421, + "missed_rescue": 965, + "avoided_harm": 469, + "selected_both_correct": 2941, + "selected_both_wrong": 315 + }, + { + "budget": 0.41, + "threshold": -2.089120388031006, + "operator": "ge", + "actual_fraction": 0.40625672669871565, + "metric": 0.8666140489344909, + "rescued": 1863, + "harmed": 436, + "missed_rescue": 928, + "avoided_harm": 454, + "selected_both_correct": 3042, + "selected_both_wrong": 321 + }, + { + "budget": 0.42, + "threshold": -2.2022628784179688, + "operator": "ge", + "actual_fraction": 0.4162301786611179, + "metric": 0.8682643323527302, + "rescued": 1900, + "harmed": 450, + "missed_rescue": 891, + "avoided_harm": 440, + "selected_both_correct": 3127, + "selected_both_wrong": 324 + }, + { + "budget": 0.43, + "threshold": -2.310732841491699, + "operator": "ge", + "actual_fraction": 0.42562961899978474, + "metric": 0.869268852694267, + "rescued": 1931, + "harmed": 467, + "missed_rescue": 860, + "avoided_harm": 423, + "selected_both_correct": 3205, + "selected_both_wrong": 329 + }, + { + "budget": 0.44, + "threshold": -2.4193384647369385, + "operator": "ge", + "actual_fraction": 0.4350290593384516, + "metric": 0.8697711128650355, + "rescued": 1952, + "harmed": 481, + "missed_rescue": 839, + "avoided_harm": 409, + "selected_both_correct": 3297, + "selected_both_wrong": 333 + }, + { + "budget": 0.45, + "threshold": -2.539083242416382, + "operator": "ge", + "actual_fraction": 0.44392623950635, + "metric": 0.8711343904714071, + "rescued": 1980, + "harmed": 490, + "missed_rescue": 811, + "avoided_harm": 400, + "selected_both_correct": 3383, + "selected_both_wrong": 334 + }, + { + "budget": 0.46, + "threshold": -2.6633739471435547, + "operator": "ge", + "actual_fraction": 0.45296692258018223, + "metric": 0.8724976680777786, + "rescued": 2006, + "harmed": 497, + "missed_rescue": 785, + "avoided_harm": 393, + "selected_both_correct": 3469, + "selected_both_wrong": 341 + }, + { + "budget": 0.47, + "threshold": -2.7973411083221436, + "operator": "ge", + "actual_fraction": 0.4627251201836837, + "metric": 0.8742914544019517, + "rescued": 2043, + "harmed": 509, + "missed_rescue": 748, + "avoided_harm": 381, + "selected_both_correct": 3552, + "selected_both_wrong": 345 + }, + { + "budget": 0.48, + "threshold": -2.923297166824341, + "operator": "ge", + "actual_fraction": 0.4711917916337806, + "metric": 0.8752242232905216, + "rescued": 2065, + "harmed": 518, + "missed_rescue": 726, + "avoided_harm": 372, + "selected_both_correct": 3637, + "selected_both_wrong": 347 + }, + { + "budget": 0.49, + "threshold": -3.0752992630004883, + "operator": "ge", + "actual_fraction": 0.48417880462079355, + "metric": 0.8763004950850255, + "rescued": 2096, + "harmed": 534, + "missed_rescue": 695, + "avoided_harm": 356, + "selected_both_correct": 3765, + "selected_both_wrong": 353 + }, + { + "budget": 0.5, + "threshold": -3.220832347869873, + "operator": "ge", + "actual_fraction": 0.4931477362416589, + "metric": 0.8769462581617278, + "rescued": 2118, + "harmed": 547, + "missed_rescue": 673, + "avoided_harm": 343, + "selected_both_correct": 3851, + "selected_both_wrong": 357 + }, + { + "budget": 0.51, + "threshold": -3.3313610553741455, + "operator": "ge", + "actual_fraction": 0.5029059338451604, + "metric": 0.8778790270502977, + "rescued": 2139, + "harmed": 555, + "missed_rescue": 652, + "avoided_harm": 335, + "selected_both_correct": 3954, + "selected_both_wrong": 361 + }, + { + "budget": 0.52, + "threshold": -3.453277587890625, + "operator": "ge", + "actual_fraction": 0.5140991605080003, + "metric": 0.8788835473918347, + "rescued": 2166, + "harmed": 568, + "missed_rescue": 625, + "avoided_harm": 322, + "selected_both_correct": 4061, + "selected_both_wrong": 370 + }, + { + "budget": 0.53, + "threshold": -3.5726635456085205, + "operator": "ge", + "actual_fraction": 0.5237138552055679, + "metric": 0.8797445648274378, + "rescued": 2185, + "harmed": 575, + "missed_rescue": 606, + "avoided_harm": 315, + "selected_both_correct": 4166, + "selected_both_wrong": 373 + }, + { + "budget": 0.54, + "threshold": -3.656648635864258, + "operator": "ge", + "actual_fraction": 0.5307454975963263, + "metric": 0.8806055822630409, + "rescued": 2203, + "harmed": 581, + "missed_rescue": 588, + "avoided_harm": 309, + "selected_both_correct": 4240, + "selected_both_wrong": 373 + }, + { + "budget": 0.55, + "threshold": -3.7778828144073486, + "operator": "ge", + "actual_fraction": 0.5407907010116955, + "metric": 0.88132309679271, + "rescued": 2222, + "harmed": 590, + "missed_rescue": 569, + "avoided_harm": 300, + "selected_both_correct": 4342, + "selected_both_wrong": 383 + }, + { + "budget": 0.56, + "threshold": -3.869736671447754, + "operator": "ge", + "actual_fraction": 0.5483963550261893, + "metric": 0.8811795938867762, + "rescued": 2229, + "harmed": 599, + "missed_rescue": 562, + "avoided_harm": 291, + "selected_both_correct": 4429, + "selected_both_wrong": 386 + }, + { + "budget": 0.57, + "threshold": -3.9598283767700195, + "operator": "ge", + "actual_fraction": 0.5584415584415584, + "metric": 0.88225586568128, + "rescued": 2254, + "harmed": 609, + "missed_rescue": 537, + "avoided_harm": 281, + "selected_both_correct": 4530, + "selected_both_wrong": 390 + }, + { + "budget": 0.58, + "threshold": -4.036509990692139, + "operator": "ge", + "actual_fraction": 0.5669799813446222, + "metric": 0.8829016287579824, + "rescued": 2267, + "harmed": 613, + "missed_rescue": 524, + "avoided_harm": 277, + "selected_both_correct": 4629, + "selected_both_wrong": 393 + }, + { + "budget": 0.59, + "threshold": -4.113898754119873, + "operator": "ge", + "actual_fraction": 0.5760206644184545, + "metric": 0.8836908947406185, + "rescued": 2282, + "harmed": 617, + "missed_rescue": 509, + "avoided_harm": 273, + "selected_both_correct": 4733, + "selected_both_wrong": 396 + }, + { + "budget": 0.6, + "threshold": -4.190845966339111, + "operator": "ge", + "actual_fraction": 0.5853483533041544, + "metric": 0.8844084092702877, + "rescued": 2302, + "harmed": 627, + "missed_rescue": 489, + "avoided_harm": 263, + "selected_both_correct": 4830, + "selected_both_wrong": 399 + }, + { + "budget": 0.61, + "threshold": -4.2746686935424805, + "operator": "ge", + "actual_fraction": 0.5960393197962258, + "metric": 0.8856999354236923, + "rescued": 2329, + "harmed": 636, + "missed_rescue": 462, + "avoided_harm": 254, + "selected_both_correct": 4939, + "selected_both_wrong": 403 + }, + { + "budget": 0.62, + "threshold": -4.353496074676514, + "operator": "ge", + "actual_fraction": 0.6075913037239004, + "metric": 0.8862021955944608, + "rescued": 2343, + "harmed": 643, + "missed_rescue": 448, + "avoided_harm": 247, + "selected_both_correct": 5077, + "selected_both_wrong": 405 + }, + { + "budget": 0.63, + "threshold": -4.4249958992004395, + "operator": "ge", + "actual_fraction": 0.6167037382506996, + "metric": 0.8861304441414939, + "rescued": 2352, + "harmed": 653, + "missed_rescue": 439, + "avoided_harm": 237, + "selected_both_correct": 5182, + "selected_both_wrong": 408 + }, + { + "budget": 0.64, + "threshold": -4.508240699768066, + "operator": "ge", + "actual_fraction": 0.6261749300423334, + "metric": 0.8869197101241301, + "rescued": 2371, + "harmed": 661, + "missed_rescue": 420, + "avoided_harm": 229, + "selected_both_correct": 5285, + "selected_both_wrong": 410 + }, + { + "budget": 0.65, + "threshold": -4.580889701843262, + "operator": "ge", + "actual_fraction": 0.6359331276458349, + "metric": 0.887780727559733, + "rescued": 2391, + "harmed": 669, + "missed_rescue": 400, + "avoided_harm": 221, + "selected_both_correct": 5384, + "selected_both_wrong": 419 + }, + { + "budget": 0.66, + "threshold": -4.649904727935791, + "operator": "ge", + "actual_fraction": 0.6443280476429648, + "metric": 0.8881394848245677, + "rescued": 2400, + "harmed": 673, + "missed_rescue": 391, + "avoided_harm": 217, + "selected_both_correct": 5484, + "selected_both_wrong": 423 + }, + { + "budget": 0.67, + "threshold": -4.736553192138672, + "operator": "ge", + "actual_fraction": 0.6554495228528378, + "metric": 0.8889287508072038, + "rescued": 2418, + "harmed": 680, + "missed_rescue": 373, + "avoided_harm": 210, + "selected_both_correct": 5611, + "selected_both_wrong": 426 + }, + { + "budget": 0.68, + "threshold": -4.816196918487549, + "operator": "ge", + "actual_fraction": 0.6679342756690823, + "metric": 0.8890722537131377, + "rescued": 2432, + "harmed": 692, + "missed_rescue": 359, + "avoided_harm": 198, + "selected_both_correct": 5754, + "selected_both_wrong": 431 + }, + { + "budget": 0.69, + "threshold": -4.892243385314941, + "operator": "ge", + "actual_fraction": 0.6774054674607161, + "metric": 0.8895745138839062, + "rescued": 2444, + "harmed": 697, + "missed_rescue": 347, + "avoided_harm": 193, + "selected_both_correct": 5868, + "selected_both_wrong": 432 + }, + { + "budget": 0.7, + "threshold": -4.959829807281494, + "operator": "ge", + "actual_fraction": 0.686733156346416, + "metric": 0.8906507856784099, + "rescued": 2466, + "harmed": 704, + "missed_rescue": 325, + "avoided_harm": 186, + "selected_both_correct": 5966, + "selected_both_wrong": 435 + }, + { + "budget": 0.71, + "threshold": -5.037147045135498, + "operator": "ge", + "actual_fraction": 0.696993614120686, + "metric": 0.8910812943962115, + "rescued": 2478, + "harmed": 710, + "missed_rescue": 313, + "avoided_harm": 180, + "selected_both_correct": 6089, + "selected_both_wrong": 437 + }, + { + "budget": 0.72, + "threshold": -5.102000713348389, + "operator": "ge", + "actual_fraction": 0.7051732797589151, + "metric": 0.891511803114013, + "rescued": 2493, + "harmed": 719, + "missed_rescue": 298, + "avoided_harm": 171, + "selected_both_correct": 6177, + "selected_both_wrong": 439 + }, + { + "budget": 0.73, + "threshold": -5.184895038604736, + "operator": "ge", + "actual_fraction": 0.7184472985577958, + "metric": 0.89251632345555, + "rescued": 2517, + "harmed": 729, + "missed_rescue": 274, + "avoided_harm": 161, + "selected_both_correct": 6327, + "selected_both_wrong": 440 + }, + { + "budget": 0.74, + "threshold": -5.24793815612793, + "operator": "ge", + "actual_fraction": 0.7277749874434958, + "metric": 0.8929468321733515, + "rescued": 2527, + "harmed": 733, + "missed_rescue": 264, + "avoided_harm": 157, + "selected_both_correct": 6442, + "selected_both_wrong": 441 + }, + { + "budget": 0.75, + "threshold": -5.323313236236572, + "operator": "ge", + "actual_fraction": 0.7382506995766664, + "metric": 0.8933773408911531, + "rescued": 2541, + "harmed": 741, + "missed_rescue": 250, + "avoided_harm": 149, + "selected_both_correct": 6560, + "selected_both_wrong": 447 + }, + { + "budget": 0.76, + "threshold": -5.399226665496826, + "operator": "ge", + "actual_fraction": 0.7480806486331348, + "metric": 0.8940948554208222, + "rescued": 2556, + "harmed": 746, + "missed_rescue": 235, + "avoided_harm": 144, + "selected_both_correct": 6674, + "selected_both_wrong": 450 + }, + { + "budget": 0.77, + "threshold": -5.4738969802856445, + "operator": "ge", + "actual_fraction": 0.7576235918777355, + "metric": 0.8950993757623592, + "rescued": 2574, + "harmed": 750, + "missed_rescue": 217, + "avoided_harm": 140, + "selected_both_correct": 6782, + "selected_both_wrong": 453 + }, + { + "budget": 0.78, + "threshold": -5.566734313964844, + "operator": "ge", + "actual_fraction": 0.7688885699935424, + "metric": 0.8957451388390615, + "rescued": 2590, + "harmed": 757, + "missed_rescue": 201, + "avoided_harm": 133, + "selected_both_correct": 6910, + "selected_both_wrong": 459 + }, + { + "budget": 0.79, + "threshold": -5.644944190979004, + "operator": "ge", + "actual_fraction": 0.7795795364856138, + "metric": 0.8958168902920284, + "rescued": 2598, + "harmed": 764, + "missed_rescue": 193, + "avoided_harm": 126, + "selected_both_correct": 7040, + "selected_both_wrong": 463 + }, + { + "budget": 0.8, + "threshold": -5.722663402557373, + "operator": "ge", + "actual_fraction": 0.791490277678123, + "metric": 0.89624739900983, + "rescued": 2611, + "harmed": 771, + "missed_rescue": 180, + "avoided_harm": 119, + "selected_both_correct": 7184, + "selected_both_wrong": 465 + }, + { + "budget": 0.81, + "threshold": -5.805805683135986, + "operator": "ge", + "actual_fraction": 0.8011049723756906, + "metric": 0.8965344048216977, + "rescued": 2621, + "harmed": 777, + "missed_rescue": 170, + "avoided_harm": 113, + "selected_both_correct": 7302, + "selected_both_wrong": 465 + }, + { + "budget": 0.82, + "threshold": -5.893346786499023, + "operator": "ge", + "actual_fraction": 0.8121546961325967, + "metric": 0.8973236708043338, + "rescued": 2638, + "harmed": 783, + "missed_rescue": 153, + "avoided_harm": 107, + "selected_both_correct": 7432, + "selected_both_wrong": 466 + }, + { + "budget": 0.83, + "threshold": -5.973089694976807, + "operator": "ge", + "actual_fraction": 0.8199756045059913, + "metric": 0.8973236708043338, + "rescued": 2643, + "harmed": 788, + "missed_rescue": 148, + "avoided_harm": 102, + "selected_both_correct": 7529, + "selected_both_wrong": 468 + }, + { + "budget": 0.84, + "threshold": -6.067188739776611, + "operator": "ge", + "actual_fraction": 0.8311688311688312, + "metric": 0.8975389251632345, + "rescued": 2654, + "harmed": 796, + "missed_rescue": 137, + "avoided_harm": 94, + "selected_both_correct": 7664, + "selected_both_wrong": 470 + }, + { + "budget": 0.85, + "threshold": -6.1676530838012695, + "operator": "ge", + "actual_fraction": 0.8439405897969434, + "metric": 0.8972519193513668, + "rescued": 2666, + "harmed": 812, + "missed_rescue": 125, + "avoided_harm": 78, + "selected_both_correct": 7812, + "selected_both_wrong": 472 + }, + { + "budget": 0.86, + "threshold": -6.258213520050049, + "operator": "ge", + "actual_fraction": 0.855349070818684, + "metric": 0.897108416445433, + "rescued": 2674, + "harmed": 822, + "missed_rescue": 117, + "avoided_harm": 68, + "selected_both_correct": 7949, + "selected_both_wrong": 476 + }, + { + "budget": 0.87, + "threshold": -6.353724002838135, + "operator": "ge", + "actual_fraction": 0.8678338236349286, + "metric": 0.8979694338810361, + "rescued": 2693, + "harmed": 829, + "missed_rescue": 98, + "avoided_harm": 61, + "selected_both_correct": 8094, + "selected_both_wrong": 479 + }, + { + "budget": 0.88, + "threshold": -6.4612135887146, + "operator": "ge", + "actual_fraction": 0.8796010619215039, + "metric": 0.8981846882399369, + "rescued": 2703, + "harmed": 836, + "missed_rescue": 88, + "avoided_harm": 54, + "selected_both_correct": 8241, + "selected_both_wrong": 479 + }, + { + "budget": 0.89, + "threshold": -6.545001983642578, + "operator": "ge", + "actual_fraction": 0.8890722537131377, + "metric": 0.898041185334003, + "rescued": 2710, + "harmed": 845, + "missed_rescue": 81, + "avoided_harm": 45, + "selected_both_correct": 8357, + "selected_both_wrong": 479 + }, + { + "budget": 0.9, + "threshold": -6.64798641204834, + "operator": "ge", + "actual_fraction": 0.898041185334003, + "metric": 0.8986151969577384, + "rescued": 2721, + "harmed": 848, + "missed_rescue": 70, + "avoided_harm": 42, + "selected_both_correct": 8467, + "selected_both_wrong": 480 + }, + { + "budget": 0.91, + "threshold": -6.747555255889893, + "operator": "ge", + "actual_fraction": 0.9070101169548683, + "metric": 0.8991174571285069, + "rescued": 2733, + "harmed": 853, + "missed_rescue": 58, + "avoided_harm": 37, + "selected_both_correct": 8572, + "selected_both_wrong": 483 + }, + { + "budget": 0.92, + "threshold": -6.860367774963379, + "operator": "ge", + "actual_fraction": 0.9184185979766091, + "metric": 0.8995479658463084, + "rescued": 2746, + "harmed": 860, + "missed_rescue": 45, + "avoided_harm": 30, + "selected_both_correct": 8709, + "selected_both_wrong": 485 + }, + { + "budget": 0.93, + "threshold": -6.962065696716309, + "operator": "ge", + "actual_fraction": 0.9269570208796728, + "metric": 0.8996914687522423, + "rescued": 2754, + "harmed": 866, + "missed_rescue": 37, + "avoided_harm": 24, + "selected_both_correct": 8813, + "selected_both_wrong": 486 + }, + { + "budget": 0.94, + "threshold": -7.099829196929932, + "operator": "ge", + "actual_fraction": 0.9389395135251489, + "metric": 0.899906723111143, + "rescued": 2761, + "harmed": 870, + "missed_rescue": 30, + "avoided_harm": 20, + "selected_both_correct": 8969, + "selected_both_wrong": 486 + }, + { + "budget": 0.95, + "threshold": -7.218595027923584, + "operator": "ge", + "actual_fraction": 0.9485542082227165, + "metric": 0.89997847456411, + "rescued": 2767, + "harmed": 875, + "missed_rescue": 24, + "avoided_harm": 15, + "selected_both_correct": 9090, + "selected_both_wrong": 488 + }, + { + "budget": 0.96, + "threshold": -7.371253967285156, + "operator": "ge", + "actual_fraction": 0.9584559087321518, + "metric": 0.9000502260170768, + "rescued": 2772, + "harmed": 879, + "missed_rescue": 19, + "avoided_harm": 11, + "selected_both_correct": 9218, + "selected_both_wrong": 489 + }, + { + "budget": 0.97, + "threshold": -7.56823205947876, + "operator": "ge", + "actual_fraction": 0.9697208868479587, + "metric": 0.9001219774700437, + "rescued": 2777, + "harmed": 883, + "missed_rescue": 14, + "avoided_harm": 7, + "selected_both_correct": 9364, + "selected_both_wrong": 491 + }, + { + "budget": 0.98, + "threshold": -7.766812801361084, + "operator": "ge", + "actual_fraction": 0.980483604792997, + "metric": 0.9001219774700437, + "rescued": 2780, + "harmed": 886, + "missed_rescue": 11, + "avoided_harm": 4, + "selected_both_correct": 9506, + "selected_both_wrong": 493 + }, + { + "budget": 0.99, + "threshold": -8.102746963500977, + "operator": "ge", + "actual_fraction": 0.9918203343617709, + "metric": 0.9004807347348783, + "rescued": 2788, + "harmed": 889, + "missed_rescue": 3, + "avoided_harm": 1, + "selected_both_correct": 9652, + "selected_both_wrong": 494 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9006242376408122, + "rescued": 2791, + "harmed": 890, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9761, + "selected_both_wrong": 495 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 4870, + "rescuable": 1449, + "harmful": 406, + "both_wrong": 257, + "unit": "sample", + "total": 6982 + }, + "test": { + "both_correct": 9761, + "rescuable": 2791, + "harmful": 890, + "both_wrong": 495, + "unit": "sample", + "total": 13937 + } + }, + "small": { + "validation": 0.7556574047550845, + "test": 0.7642247255506924 + }, + "large": { + "validation": 0.9050415353766829, + "test": 0.9006242376408122 + }, + "learned_minus_confidence_test": 0.00035875726483469705, + "learned_minus_small_test": 0.13575374901341752, + "learned_minus_large_test": -0.0006457630767022771 + }, + { + "key": [ + "construction", + "yolo26x", + "MiniCPM" + ], + "expert_display": "YOLO26x", + "run": "shared_12_yolo26x_MiniCPM", + "validation_rows": 701, + "test_rows": 3004, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detection_count", + "has_detections", + "max_score", + "mean_score", + "detection_count_valid", + "has_detections_valid", + "max_score_valid", + "mean_score_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/yolo26x/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 94, + "validation_area_0_25": 0.1745164052515915, + "validation_point": { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.22641590025257546, + "rescued": 272, + "harmed": 478, + "missed_rescue": 14, + "avoided_harm": 40, + "selected_both_correct": 1782, + "selected_both_wrong": 100 + }, + "test_point": { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9254327563249002, + "metric": 0.2523047077304935, + "rescued": 1113, + "harmed": 1932, + "missed_rescue": 85, + "avoided_harm": 177, + "selected_both_correct": 7597, + "selected_both_wrong": 478 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": 1.0, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.1583943405474877, + "rescued": 0, + "harmed": 88, + "missed_rescue": 286, + "avoided_harm": 430, + "selected_both_correct": 444, + "selected_both_wrong": 0 + }, + { + "budget": 0.2, + "threshold": -0.0010652542114257812, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.15744253541872588, + "rescued": 0, + "harmed": 94, + "missed_rescue": 286, + "avoided_harm": 424, + "selected_both_correct": 466, + "selected_both_wrong": 0 + }, + { + "budget": 0.21, + "threshold": -0.0011425018310546875, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.1562524865065214, + "rescued": 0, + "harmed": 100, + "missed_rescue": 286, + "avoided_harm": 418, + "selected_both_correct": 488, + "selected_both_wrong": 0 + }, + { + "budget": 0.22, + "threshold": -0.0011968612670898438, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.15524778086857327, + "rescued": 0, + "harmed": 106, + "missed_rescue": 286, + "avoided_harm": 412, + "selected_both_correct": 509, + "selected_both_wrong": 1 + }, + { + "budget": 0.23, + "threshold": -0.0012493133544921875, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.1542743955978643, + "rescued": 0, + "harmed": 112, + "missed_rescue": 286, + "avoided_harm": 406, + "selected_both_correct": 531, + "selected_both_wrong": 1 + }, + { + "budget": 0.24, + "threshold": -0.0013513565063476562, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.15300917812583928, + "rescued": 0, + "harmed": 118, + "missed_rescue": 286, + "avoided_harm": 400, + "selected_both_correct": 553, + "selected_both_wrong": 1 + }, + { + "budget": 0.25, + "threshold": -0.0015010833740234375, + "operator": "gt", + "actual_fraction": 0.24821683309557774, + "metric": 0.15249392985085833, + "rescued": 0, + "harmed": 122, + "missed_rescue": 286, + "avoided_harm": 396, + "selected_both_correct": 572, + "selected_both_wrong": 2 + }, + { + "budget": 0.26, + "threshold": -0.00157928466796875, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.15123140165156973, + "rescued": 0, + "harmed": 131, + "missed_rescue": 286, + "avoided_harm": 387, + "selected_both_correct": 595, + "selected_both_wrong": 2 + }, + { + "budget": 0.27, + "threshold": -0.0017137527465820312, + "operator": "gt", + "actual_fraction": 0.268188302425107, + "metric": 0.15063802192285164, + "rescued": 0, + "harmed": 136, + "missed_rescue": 286, + "avoided_harm": 382, + "selected_both_correct": 614, + "selected_both_wrong": 2 + }, + { + "budget": 0.28, + "threshold": -0.0017614364624023438, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.14987516228902426, + "rescued": 0, + "harmed": 143, + "missed_rescue": 286, + "avoided_harm": 375, + "selected_both_correct": 639, + "selected_both_wrong": 2 + }, + { + "budget": 0.29, + "threshold": -0.0018463134765625, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.14942913195896182, + "rescued": 0, + "harmed": 147, + "missed_rescue": 286, + "avoided_harm": 371, + "selected_both_correct": 663, + "selected_both_wrong": 2 + }, + { + "budget": 0.3, + "threshold": -0.001934051513671875, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.14874977232993136, + "rescued": 0, + "harmed": 152, + "missed_rescue": 286, + "avoided_harm": 366, + "selected_both_correct": 686, + "selected_both_wrong": 2 + }, + { + "budget": 0.31, + "threshold": -0.0020427703857421875, + "operator": "gt", + "actual_fraction": 0.30813124108416545, + "metric": 0.1481550075018755, + "rescued": 0, + "harmed": 156, + "missed_rescue": 286, + "avoided_harm": 362, + "selected_both_correct": 706, + "selected_both_wrong": 2 + }, + { + "budget": 0.32, + "threshold": -0.0021495819091796875, + "operator": "gt", + "actual_fraction": 0.3181169757489301, + "metric": 0.14737954730376035, + "rescued": 0, + "harmed": 162, + "missed_rescue": 286, + "avoided_harm": 356, + "selected_both_correct": 728, + "selected_both_wrong": 2 + }, + { + "budget": 0.33, + "threshold": -0.002208709716796875, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.14624628747548335, + "rescued": 0, + "harmed": 170, + "missed_rescue": 286, + "avoided_harm": 348, + "selected_both_correct": 752, + "selected_both_wrong": 2 + }, + { + "budget": 0.34, + "threshold": -0.0023326873779296875, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.14552266133559724, + "rescued": 0, + "harmed": 176, + "missed_rescue": 286, + "avoided_harm": 342, + "selected_both_correct": 774, + "selected_both_wrong": 2 + }, + { + "budget": 0.35, + "threshold": -0.0024623870849609375, + "operator": "gt", + "actual_fraction": 0.34807417974322397, + "metric": 0.14490257033424028, + "rescued": 0, + "harmed": 181, + "missed_rescue": 286, + "avoided_harm": 337, + "selected_both_correct": 793, + "selected_both_wrong": 2 + }, + { + "budget": 0.36, + "threshold": -0.0025615692138671875, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.1443034349839422, + "rescued": 0, + "harmed": 188, + "missed_rescue": 286, + "avoided_harm": 330, + "selected_both_correct": 818, + "selected_both_wrong": 2 + }, + { + "budget": 0.37, + "threshold": -0.0026721954345703125, + "operator": "gt", + "actual_fraction": 0.3666191155492154, + "metric": 0.14397079647073363, + "rescued": 0, + "harmed": 192, + "missed_rescue": 286, + "avoided_harm": 326, + "selected_both_correct": 834, + "selected_both_wrong": 2 + }, + { + "budget": 0.38, + "threshold": -0.0028228759765625, + "operator": "gt", + "actual_fraction": 0.37660485021398005, + "metric": 0.14328749409772273, + "rescued": 0, + "harmed": 198, + "missed_rescue": 286, + "avoided_harm": 320, + "selected_both_correct": 856, + "selected_both_wrong": 2 + }, + { + "budget": 0.39, + "threshold": -0.0029354095458984375, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.14535108686369555, + "rescued": 1, + "harmed": 209, + "missed_rescue": 285, + "avoided_harm": 309, + "selected_both_correct": 880, + "selected_both_wrong": 2 + }, + { + "budget": 0.4, + "threshold": -0.00307464599609375, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.14474378206514643, + "rescued": 1, + "harmed": 214, + "missed_rescue": 285, + "avoided_harm": 304, + "selected_both_correct": 903, + "selected_both_wrong": 2 + }, + { + "budget": 0.41, + "threshold": -0.0033626556396484375, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.14399177962039386, + "rescued": 1, + "harmed": 221, + "missed_rescue": 285, + "avoided_harm": 297, + "selected_both_correct": 924, + "selected_both_wrong": 2 + }, + { + "budget": 0.42, + "threshold": -0.0035381317138671875, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.14353714231992915, + "rescued": 1, + "harmed": 227, + "missed_rescue": 285, + "avoided_harm": 291, + "selected_both_correct": 946, + "selected_both_wrong": 2 + }, + { + "budget": 0.43, + "threshold": -0.003765106201171875, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.1430174966224799, + "rescued": 1, + "harmed": 234, + "missed_rescue": 285, + "avoided_harm": 284, + "selected_both_correct": 967, + "selected_both_wrong": 2 + }, + { + "budget": 0.44, + "threshold": -0.00392913818359375, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.14225189964081136, + "rescued": 1, + "harmed": 242, + "missed_rescue": 285, + "avoided_harm": 276, + "selected_both_correct": 987, + "selected_both_wrong": 2 + }, + { + "budget": 0.45, + "threshold": -0.004085540771484375, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.14143790301195314, + "rescued": 1, + "harmed": 248, + "missed_rescue": 285, + "avoided_harm": 270, + "selected_both_correct": 1009, + "selected_both_wrong": 2 + }, + { + "budget": 0.46, + "threshold": -0.004364013671875, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.14328380481065045, + "rescued": 2, + "harmed": 254, + "missed_rescue": 284, + "avoided_harm": 264, + "selected_both_correct": 1030, + "selected_both_wrong": 2 + }, + { + "budget": 0.47, + "threshold": -0.004680633544921875, + "operator": "gt", + "actual_fraction": 0.46790299572039945, + "metric": 0.145146248248267, + "rescued": 3, + "harmed": 259, + "missed_rescue": 283, + "avoided_harm": 259, + "selected_both_correct": 1048, + "selected_both_wrong": 2 + }, + { + "budget": 0.48, + "threshold": -0.00490570068359375, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.14408640672685616, + "rescued": 3, + "harmed": 268, + "missed_rescue": 283, + "avoided_harm": 250, + "selected_both_correct": 1071, + "selected_both_wrong": 2 + }, + { + "budget": 0.49, + "threshold": -0.005039215087890625, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.14363247863247863, + "rescued": 5, + "harmed": 274, + "missed_rescue": 281, + "avoided_harm": 244, + "selected_both_correct": 1090, + "selected_both_wrong": 3 + }, + { + "budget": 0.5, + "threshold": -0.005321502685546875, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.14422472705491574, + "rescued": 8, + "harmed": 281, + "missed_rescue": 278, + "avoided_harm": 237, + "selected_both_correct": 1104, + "selected_both_wrong": 7 + }, + { + "budget": 0.51, + "threshold": -0.005619049072265625, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.14584304986884322, + "rescued": 15, + "harmed": 285, + "missed_rescue": 271, + "avoided_harm": 233, + "selected_both_correct": 1120, + "selected_both_wrong": 8 + }, + { + "budget": 0.52, + "threshold": -0.00586700439453125, + "operator": "gt", + "actual_fraction": 0.5178316690442225, + "metric": 0.14475873599710132, + "rescued": 18, + "harmed": 291, + "missed_rescue": 268, + "avoided_harm": 227, + "selected_both_correct": 1133, + "selected_both_wrong": 10 + }, + { + "budget": 0.53, + "threshold": -0.00605010986328125, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.14599834427394986, + "rescued": 24, + "harmed": 297, + "missed_rescue": 262, + "avoided_harm": 221, + "selected_both_correct": 1151, + "selected_both_wrong": 12 + }, + { + "budget": 0.54, + "threshold": -0.006439208984375, + "operator": "gt", + "actual_fraction": 0.5378031383737518, + "metric": 0.14694159872855866, + "rescued": 29, + "harmed": 301, + "missed_rescue": 257, + "avoided_harm": 217, + "selected_both_correct": 1166, + "selected_both_wrong": 12 + }, + { + "budget": 0.55, + "threshold": -0.006641387939453125, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.14684487529484538, + "rescued": 35, + "harmed": 305, + "missed_rescue": 251, + "avoided_harm": 213, + "selected_both_correct": 1187, + "selected_both_wrong": 13 + }, + { + "budget": 0.56, + "threshold": -0.0070648193359375, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.14849467509528708, + "rescued": 42, + "harmed": 308, + "missed_rescue": 244, + "avoided_harm": 210, + "selected_both_correct": 1203, + "selected_both_wrong": 15 + }, + { + "budget": 0.57, + "threshold": -0.007518768310546875, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.1496557181286853, + "rescued": 48, + "harmed": 311, + "missed_rescue": 238, + "avoided_harm": 207, + "selected_both_correct": 1221, + "selected_both_wrong": 16 + }, + { + "budget": 0.58, + "threshold": -0.00818634033203125, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.15076631577130084, + "rescued": 53, + "harmed": 315, + "missed_rescue": 233, + "avoided_harm": 203, + "selected_both_correct": 1237, + "selected_both_wrong": 19 + }, + { + "budget": 0.59, + "threshold": -0.00878143310546875, + "operator": "gt", + "actual_fraction": 0.5877318116975749, + "metric": 0.15169442750270445, + "rescued": 57, + "harmed": 320, + "missed_rescue": 229, + "avoided_harm": 198, + "selected_both_correct": 1250, + "selected_both_wrong": 21 + }, + { + "budget": 0.6, + "threshold": -0.00955963134765625, + "operator": "gt", + "actual_fraction": 0.5977175463623395, + "metric": 0.1523976411340631, + "rescued": 62, + "harmed": 322, + "missed_rescue": 224, + "avoided_harm": 196, + "selected_both_correct": 1268, + "selected_both_wrong": 24 + }, + { + "budget": 0.61, + "threshold": -0.0099334716796875, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.15402113497563988, + "rescued": 68, + "harmed": 328, + "missed_rescue": 218, + "avoided_harm": 190, + "selected_both_correct": 1286, + "selected_both_wrong": 26 + }, + { + "budget": 0.62, + "threshold": -0.010650634765625, + "operator": "gt", + "actual_fraction": 0.6176890156918687, + "metric": 0.1533732028512746, + "rescued": 71, + "harmed": 331, + "missed_rescue": 215, + "avoided_harm": 187, + "selected_both_correct": 1302, + "selected_both_wrong": 28 + }, + { + "budget": 0.63, + "threshold": -0.01129150390625, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.15514650916956785, + "rescued": 79, + "harmed": 337, + "missed_rescue": 207, + "avoided_harm": 181, + "selected_both_correct": 1317, + "selected_both_wrong": 31 + }, + { + "budget": 0.64, + "threshold": -0.011871337890625, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.1583928135738368, + "rescued": 85, + "harmed": 341, + "missed_rescue": 201, + "avoided_harm": 177, + "selected_both_correct": 1332, + "selected_both_wrong": 34 + }, + { + "budget": 0.65, + "threshold": -0.0125274658203125, + "operator": "gt", + "actual_fraction": 0.6476462196861626, + "metric": 0.15975557497296627, + "rescued": 92, + "harmed": 346, + "missed_rescue": 194, + "avoided_harm": 172, + "selected_both_correct": 1344, + "selected_both_wrong": 34 + }, + { + "budget": 0.66, + "threshold": -0.012969970703125, + "operator": "gt", + "actual_fraction": 0.6562054208273894, + "metric": 0.16074742775562448, + "rescued": 95, + "harmed": 347, + "missed_rescue": 191, + "avoided_harm": 171, + "selected_both_correct": 1361, + "selected_both_wrong": 37 + }, + { + "budget": 0.67, + "threshold": -0.01363372802734375, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.16246869830774383, + "rescued": 102, + "harmed": 352, + "missed_rescue": 184, + "avoided_harm": 166, + "selected_both_correct": 1381, + "selected_both_wrong": 41 + }, + { + "budget": 0.68, + "threshold": -0.01473236083984375, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.16459532061840704, + "rescued": 109, + "harmed": 357, + "missed_rescue": 177, + "avoided_harm": 161, + "selected_both_correct": 1396, + "selected_both_wrong": 42 + }, + { + "budget": 0.69, + "threshold": -0.015716552734375, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.16603896761133602, + "rescued": 114, + "harmed": 362, + "missed_rescue": 172, + "avoided_harm": 156, + "selected_both_correct": 1411, + "selected_both_wrong": 45 + }, + { + "budget": 0.7, + "threshold": -0.017578125, + "operator": "gt", + "actual_fraction": 0.6975748930099858, + "metric": 0.16707723655265538, + "rescued": 119, + "harmed": 366, + "missed_rescue": 167, + "avoided_harm": 152, + "selected_both_correct": 1423, + "selected_both_wrong": 48 + }, + { + "budget": 0.71, + "threshold": -0.0189056396484375, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.16940425786435637, + "rescued": 126, + "harmed": 371, + "missed_rescue": 160, + "avoided_harm": 147, + "selected_both_correct": 1441, + "selected_both_wrong": 50 + }, + { + "budget": 0.72, + "threshold": -0.019866943359375, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.1708064509820028, + "rescued": 134, + "harmed": 379, + "missed_rescue": 152, + "avoided_harm": 139, + "selected_both_correct": 1453, + "selected_both_wrong": 50 + }, + { + "budget": 0.73, + "threshold": -0.020965576171875, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.17255814544390938, + "rescued": 139, + "harmed": 381, + "missed_rescue": 147, + "avoided_harm": 137, + "selected_both_correct": 1470, + "selected_both_wrong": 54 + }, + { + "budget": 0.74, + "threshold": -0.021453857421875, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.17530145923526386, + "rescued": 146, + "harmed": 384, + "missed_rescue": 140, + "avoided_harm": 134, + "selected_both_correct": 1487, + "selected_both_wrong": 55 + }, + { + "budget": 0.75, + "threshold": -0.0248870849609375, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.17900218396288498, + "rescued": 155, + "harmed": 389, + "missed_rescue": 131, + "avoided_harm": 129, + "selected_both_correct": 1499, + "selected_both_wrong": 57 + }, + { + "budget": 0.76, + "threshold": -0.0269622802734375, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.18402223394204145, + "rescued": 163, + "harmed": 392, + "missed_rescue": 123, + "avoided_harm": 126, + "selected_both_correct": 1514, + "selected_both_wrong": 59 + }, + { + "budget": 0.77, + "threshold": -0.0282135009765625, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.18675324648763036, + "rescued": 169, + "harmed": 395, + "missed_rescue": 117, + "avoided_harm": 123, + "selected_both_correct": 1531, + "selected_both_wrong": 61 + }, + { + "budget": 0.78, + "threshold": -0.0299835205078125, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.18924798946215277, + "rescued": 175, + "harmed": 399, + "missed_rescue": 111, + "avoided_harm": 119, + "selected_both_correct": 1547, + "selected_both_wrong": 63 + }, + { + "budget": 0.79, + "threshold": -0.033966064453125, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.19150730156013804, + "rescued": 181, + "harmed": 403, + "missed_rescue": 105, + "avoided_harm": 115, + "selected_both_correct": 1562, + "selected_both_wrong": 66 + }, + { + "budget": 0.8, + "threshold": -0.037200927734375, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.19267082077365763, + "rescued": 185, + "harmed": 410, + "missed_rescue": 101, + "avoided_harm": 108, + "selected_both_correct": 1575, + "selected_both_wrong": 70 + }, + { + "budget": 0.81, + "threshold": -0.04132080078125, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.1985395820585592, + "rescued": 193, + "harmed": 413, + "missed_rescue": 93, + "avoided_harm": 105, + "selected_both_correct": 1589, + "selected_both_wrong": 73 + }, + { + "budget": 0.82, + "threshold": -0.0455322265625, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.20050380794511347, + "rescued": 198, + "harmed": 417, + "missed_rescue": 88, + "avoided_harm": 101, + "selected_both_correct": 1605, + "selected_both_wrong": 76 + }, + { + "budget": 0.83, + "threshold": -0.051849365234375, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.20479272888770159, + "rescued": 205, + "harmed": 422, + "missed_rescue": 81, + "avoided_harm": 96, + "selected_both_correct": 1619, + "selected_both_wrong": 78 + }, + { + "budget": 0.84, + "threshold": -0.06878662109375, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.20996516586206454, + "rescued": 211, + "harmed": 428, + "missed_rescue": 75, + "avoided_harm": 90, + "selected_both_correct": 1631, + "selected_both_wrong": 82 + }, + { + "budget": 0.85, + "threshold": -0.08477783203125, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.2153593554624366, + "rescued": 218, + "harmed": 430, + "missed_rescue": 68, + "avoided_harm": 88, + "selected_both_correct": 1647, + "selected_both_wrong": 85 + }, + { + "budget": 0.86, + "threshold": -0.09912109375, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.20675118175118173, + "rescued": 225, + "harmed": 439, + "missed_rescue": 61, + "avoided_harm": 79, + "selected_both_correct": 1658, + "selected_both_wrong": 86 + }, + { + "budget": 0.87, + "threshold": -0.1298828125, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.2103979829447119, + "rescued": 232, + "harmed": 441, + "missed_rescue": 54, + "avoided_harm": 77, + "selected_both_correct": 1675, + "selected_both_wrong": 88 + }, + { + "budget": 0.88, + "threshold": -0.1441650390625, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.2127029758032997, + "rescued": 236, + "harmed": 447, + "missed_rescue": 50, + "avoided_harm": 71, + "selected_both_correct": 1691, + "selected_both_wrong": 90 + }, + { + "budget": 0.89, + "threshold": -0.182373046875, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.21408261584117613, + "rescued": 244, + "harmed": 453, + "missed_rescue": 42, + "avoided_harm": 65, + "selected_both_correct": 1705, + "selected_both_wrong": 90 + }, + { + "budget": 0.9, + "threshold": -0.2049560546875, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.21447015425888666, + "rescued": 251, + "harmed": 460, + "missed_rescue": 35, + "avoided_harm": 58, + "selected_both_correct": 1716, + "selected_both_wrong": 93 + }, + { + "budget": 0.91, + "threshold": -0.25830078125, + "operator": "gt", + "actual_fraction": 0.9072753209700428, + "metric": 0.21778616982331747, + "rescued": 255, + "harmed": 462, + "missed_rescue": 31, + "avoided_harm": 56, + "selected_both_correct": 1730, + "selected_both_wrong": 97 + }, + { + "budget": 0.92, + "threshold": -0.32080078125, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.22341697071831912, + "rescued": 265, + "harmed": 468, + "missed_rescue": 21, + "avoided_harm": 50, + "selected_both_correct": 1746, + "selected_both_wrong": 97 + }, + { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2260324221654167, + "rescued": 269, + "harmed": 475, + "missed_rescue": 17, + "avoided_harm": 43, + "selected_both_correct": 1762, + "selected_both_wrong": 98 + }, + { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.22641590025257546, + "rescued": 272, + "harmed": 478, + "missed_rescue": 14, + "avoided_harm": 40, + "selected_both_correct": 1782, + "selected_both_wrong": 100 + }, + { + "budget": 0.95, + "threshold": -0.611328125, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.2213594019034288, + "rescued": 273, + "harmed": 486, + "missed_rescue": 13, + "avoided_harm": 32, + "selected_both_correct": 1799, + "selected_both_wrong": 102 + }, + { + "budget": 0.96, + "threshold": -0.70068359375, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.21682778770996153, + "rescued": 278, + "harmed": 495, + "missed_rescue": 8, + "avoided_harm": 23, + "selected_both_correct": 1813, + "selected_both_wrong": 102 + }, + { + "budget": 0.97, + "threshold": -0.75048828125, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.21297215749387616, + "rescued": 280, + "harmed": 500, + "missed_rescue": 6, + "avoided_harm": 18, + "selected_both_correct": 1831, + "selected_both_wrong": 105 + }, + { + "budget": 0.98, + "threshold": -0.7841796875, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.21025141881293422, + "rescued": 282, + "harmed": 504, + "missed_rescue": 4, + "avoided_harm": 14, + "selected_both_correct": 1851, + "selected_both_wrong": 107 + }, + { + "budget": 0.99, + "threshold": -0.85107421875, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.20718702865761687, + "rescued": 284, + "harmed": 512, + "missed_rescue": 2, + "avoided_harm": 6, + "selected_both_correct": 1869, + "selected_both_wrong": 107 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.20666016863629594, + "rescued": 286, + "harmed": 518, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 1891, + "selected_both_wrong": 109 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": 1.0, + "operator": "ge", + "actual_fraction": 0.17942743009320905, + "metric": 0.16785638110940382, + "rescued": 3, + "harmed": 361, + "missed_rescue": 1195, + "avoided_harm": 1748, + "selected_both_correct": 1790, + "selected_both_wrong": 2 + }, + { + "budget": 0.2, + "threshold": -0.0010652542114257812, + "operator": "ge", + "actual_fraction": 0.19340878828229027, + "metric": 0.1667569975208824, + "rescued": 3, + "harmed": 392, + "missed_rescue": 1195, + "avoided_harm": 1717, + "selected_both_correct": 1927, + "selected_both_wrong": 2 + }, + { + "budget": 0.21, + "threshold": -0.0011425018310546875, + "operator": "ge", + "actual_fraction": 0.20306258322237017, + "metric": 0.16670205561344506, + "rescued": 4, + "harmed": 415, + "missed_rescue": 1194, + "avoided_harm": 1694, + "selected_both_correct": 2018, + "selected_both_wrong": 3 + }, + { + "budget": 0.22, + "threshold": -0.0011968612670898438, + "operator": "ge", + "actual_fraction": 0.20838881491344874, + "metric": 0.16636966208367632, + "rescued": 4, + "harmed": 426, + "missed_rescue": 1194, + "avoided_harm": 1683, + "selected_both_correct": 2071, + "selected_both_wrong": 3 + }, + { + "budget": 0.23, + "threshold": -0.0012493133544921875, + "operator": "ge", + "actual_fraction": 0.21537949400798934, + "metric": 0.1674861536267502, + "rescued": 5, + "harmed": 440, + "missed_rescue": 1193, + "avoided_harm": 1669, + "selected_both_correct": 2140, + "selected_both_wrong": 3 + }, + { + "budget": 0.24, + "threshold": -0.0013513565063476562, + "operator": "ge", + "actual_fraction": 0.23135818908122505, + "metric": 0.1659144502174876, + "rescued": 5, + "harmed": 482, + "missed_rescue": 1193, + "avoided_harm": 1627, + "selected_both_correct": 2289, + "selected_both_wrong": 4 + }, + { + "budget": 0.25, + "threshold": -0.0015010833740234375, + "operator": "gt", + "actual_fraction": 0.24800266311584554, + "metric": 0.16716244725738397, + "rescued": 7, + "harmed": 526, + "missed_rescue": 1191, + "avoided_harm": 1583, + "selected_both_correct": 2441, + "selected_both_wrong": 6 + }, + { + "budget": 0.26, + "threshold": -0.00157928466796875, + "operator": "ge", + "actual_fraction": 0.2589880159786951, + "metric": 0.1678664772207598, + "rescued": 8, + "harmed": 550, + "missed_rescue": 1190, + "avoided_harm": 1559, + "selected_both_correct": 2548, + "selected_both_wrong": 6 + }, + { + "budget": 0.27, + "threshold": -0.0017137527465820312, + "operator": "gt", + "actual_fraction": 0.27430093209054596, + "metric": 0.16649682507763983, + "rescued": 8, + "harmed": 594, + "missed_rescue": 1190, + "avoided_harm": 1515, + "selected_both_correct": 2688, + "selected_both_wrong": 6 + }, + { + "budget": 0.28, + "threshold": -0.0017614364624023438, + "operator": "ge", + "actual_fraction": 0.281624500665779, + "metric": 0.16581256695877195, + "rescued": 8, + "harmed": 621, + "missed_rescue": 1190, + "avoided_harm": 1488, + "selected_both_correct": 2749, + "selected_both_wrong": 6 + }, + { + "budget": 0.29, + "threshold": -0.0018463134765625, + "operator": "ge", + "actual_fraction": 0.28761651131824234, + "metric": 0.1651274175371034, + "rescued": 8, + "harmed": 636, + "missed_rescue": 1190, + "avoided_harm": 1473, + "selected_both_correct": 2806, + "selected_both_wrong": 6 + }, + { + "budget": 0.3, + "threshold": -0.001934051513671875, + "operator": "ge", + "actual_fraction": 0.2969374167776298, + "metric": 0.16471286313286057, + "rescued": 8, + "harmed": 657, + "missed_rescue": 1190, + "avoided_harm": 1452, + "selected_both_correct": 2897, + "selected_both_wrong": 6 + }, + { + "budget": 0.31, + "threshold": -0.0020427703857421875, + "operator": "gt", + "actual_fraction": 0.30459387483355527, + "metric": 0.16408718171666042, + "rescued": 8, + "harmed": 676, + "missed_rescue": 1190, + "avoided_harm": 1433, + "selected_both_correct": 2969, + "selected_both_wrong": 7 + }, + { + "budget": 0.32, + "threshold": -0.0021495819091796875, + "operator": "gt", + "actual_fraction": 0.3129161118508655, + "metric": 0.1636354375797962, + "rescued": 9, + "harmed": 701, + "missed_rescue": 1189, + "avoided_harm": 1408, + "selected_both_correct": 3043, + "selected_both_wrong": 7 + }, + { + "budget": 0.33, + "threshold": -0.002208709716796875, + "operator": "ge", + "actual_fraction": 0.31990679094540614, + "metric": 0.16348828966450218, + "rescued": 10, + "harmed": 720, + "missed_rescue": 1188, + "avoided_harm": 1389, + "selected_both_correct": 3107, + "selected_both_wrong": 7 + }, + { + "budget": 0.34, + "threshold": -0.0023326873779296875, + "operator": "ge", + "actual_fraction": 0.3322237017310253, + "metric": 0.16484645578128018, + "rescued": 12, + "harmed": 749, + "missed_rescue": 1186, + "avoided_harm": 1360, + "selected_both_correct": 3224, + "selected_both_wrong": 7 + }, + { + "budget": 0.35, + "threshold": -0.0024623870849609375, + "operator": "gt", + "actual_fraction": 0.34054593874833555, + "metric": 0.16526596702267776, + "rescued": 13, + "harmed": 767, + "missed_rescue": 1185, + "avoided_harm": 1342, + "selected_both_correct": 3304, + "selected_both_wrong": 8 + }, + { + "budget": 0.36, + "threshold": -0.0025615692138671875, + "operator": "ge", + "actual_fraction": 0.3488681757656458, + "metric": 0.16520260386826902, + "rescued": 14, + "harmed": 792, + "missed_rescue": 1184, + "avoided_harm": 1317, + "selected_both_correct": 3378, + "selected_both_wrong": 8 + }, + { + "budget": 0.37, + "threshold": -0.0026721954345703125, + "operator": "gt", + "actual_fraction": 0.3581890812250333, + "metric": 0.16665740832732326, + "rescued": 17, + "harmed": 812, + "missed_rescue": 1181, + "avoided_harm": 1297, + "selected_both_correct": 3467, + "selected_both_wrong": 8 + }, + { + "budget": 0.38, + "threshold": -0.0028228759765625, + "operator": "gt", + "actual_fraction": 0.37017310252996005, + "metric": 0.1659164559036785, + "rescued": 17, + "harmed": 844, + "missed_rescue": 1181, + "avoided_harm": 1265, + "selected_both_correct": 3579, + "selected_both_wrong": 8 + }, + { + "budget": 0.39, + "threshold": -0.0029354095458984375, + "operator": "ge", + "actual_fraction": 0.37683089214380827, + "metric": 0.16604558576074954, + "rescued": 18, + "harmed": 868, + "missed_rescue": 1180, + "avoided_harm": 1241, + "selected_both_correct": 3633, + "selected_both_wrong": 9 + }, + { + "budget": 0.4, + "threshold": -0.00307464599609375, + "operator": "ge", + "actual_fraction": 0.38348868175765644, + "metric": 0.16627533009733186, + "rescued": 19, + "harmed": 882, + "missed_rescue": 1179, + "avoided_harm": 1227, + "selected_both_correct": 3698, + "selected_both_wrong": 9 + }, + { + "budget": 0.41, + "threshold": -0.0033626556396484375, + "operator": "ge", + "actual_fraction": 0.4021304926764314, + "metric": 0.16596184167496, + "rescued": 21, + "harmed": 925, + "missed_rescue": 1177, + "avoided_harm": 1184, + "selected_both_correct": 3876, + "selected_both_wrong": 10 + }, + { + "budget": 0.42, + "threshold": -0.0035381317138671875, + "operator": "ge", + "actual_fraction": 0.41311584553928093, + "metric": 0.16543665040595484, + "rescued": 22, + "harmed": 952, + "missed_rescue": 1176, + "avoided_harm": 1157, + "selected_both_correct": 3980, + "selected_both_wrong": 10 + }, + { + "budget": 0.43, + "threshold": -0.003765106201171875, + "operator": "ge", + "actual_fraction": 0.42676431424766975, + "metric": 0.16433504278528627, + "rescued": 22, + "harmed": 996, + "missed_rescue": 1176, + "avoided_harm": 1113, + "selected_both_correct": 4100, + "selected_both_wrong": 10 + }, + { + "budget": 0.44, + "threshold": -0.00392913818359375, + "operator": "ge", + "actual_fraction": 0.4324234354194407, + "metric": 0.1640338494529538, + "rescued": 22, + "harmed": 1010, + "missed_rescue": 1176, + "avoided_harm": 1099, + "selected_both_correct": 4154, + "selected_both_wrong": 10 + }, + { + "budget": 0.45, + "threshold": -0.004085540771484375, + "operator": "ge", + "actual_fraction": 0.44107856191744343, + "metric": 0.16355335117560724, + "rescued": 22, + "harmed": 1030, + "missed_rescue": 1176, + "avoided_harm": 1079, + "selected_both_correct": 4238, + "selected_both_wrong": 10 + }, + { + "budget": 0.46, + "threshold": -0.004364013671875, + "operator": "ge", + "actual_fraction": 0.45772303595206393, + "metric": 0.16328618395901012, + "rescued": 25, + "harmed": 1072, + "missed_rescue": 1173, + "avoided_harm": 1037, + "selected_both_correct": 4393, + "selected_both_wrong": 10 + }, + { + "budget": 0.47, + "threshold": -0.004680633544921875, + "operator": "gt", + "actual_fraction": 0.4717043941411451, + "metric": 0.16343963675474008, + "rescued": 27, + "harmed": 1106, + "missed_rescue": 1171, + "avoided_harm": 1003, + "selected_both_correct": 4524, + "selected_both_wrong": 11 + }, + { + "budget": 0.48, + "threshold": -0.00490570068359375, + "operator": "ge", + "actual_fraction": 0.4826897470039947, + "metric": 0.16288089231855363, + "rescued": 27, + "harmed": 1135, + "missed_rescue": 1171, + "avoided_harm": 974, + "selected_both_correct": 4626, + "selected_both_wrong": 12 + }, + { + "budget": 0.49, + "threshold": -0.005039215087890625, + "operator": "ge", + "actual_fraction": 0.4896804260985353, + "metric": 0.16253980428982567, + "rescued": 28, + "harmed": 1150, + "missed_rescue": 1170, + "avoided_harm": 959, + "selected_both_correct": 4691, + "selected_both_wrong": 15 + }, + { + "budget": 0.5, + "threshold": -0.005321502685546875, + "operator": "ge", + "actual_fraction": 0.5013315579227696, + "metric": 0.16396334310527208, + "rescued": 56, + "harmed": 1172, + "missed_rescue": 1142, + "avoided_harm": 937, + "selected_both_correct": 4775, + "selected_both_wrong": 21 + }, + { + "budget": 0.51, + "threshold": -0.005619049072265625, + "operator": "ge", + "actual_fraction": 0.5116511318242344, + "metric": 0.16505546455344486, + "rescued": 76, + "harmed": 1183, + "missed_rescue": 1122, + "avoided_harm": 926, + "selected_both_correct": 4858, + "selected_both_wrong": 31 + }, + { + "budget": 0.52, + "threshold": -0.00586700439453125, + "operator": "gt", + "actual_fraction": 0.5183089214380826, + "metric": 0.16558890552522137, + "rescued": 91, + "harmed": 1197, + "missed_rescue": 1107, + "avoided_harm": 912, + "selected_both_correct": 4904, + "selected_both_wrong": 36 + }, + { + "budget": 0.53, + "threshold": -0.00605010986328125, + "operator": "ge", + "actual_fraction": 0.5252996005326231, + "metric": 0.1665272377685939, + "rescued": 107, + "harmed": 1214, + "missed_rescue": 1091, + "avoided_harm": 895, + "selected_both_correct": 4950, + "selected_both_wrong": 41 + }, + { + "budget": 0.54, + "threshold": -0.006439208984375, + "operator": "gt", + "actual_fraction": 0.5396138482023968, + "metric": 0.16808344009752285, + "rescued": 144, + "harmed": 1241, + "missed_rescue": 1054, + "avoided_harm": 868, + "selected_both_correct": 5049, + "selected_both_wrong": 50 + }, + { + "budget": 0.55, + "threshold": -0.006641387939453125, + "operator": "ge", + "actual_fraction": 0.5452729693741678, + "metric": 0.16859171218698246, + "rescued": 156, + "harmed": 1251, + "missed_rescue": 1042, + "avoided_harm": 858, + "selected_both_correct": 5088, + "selected_both_wrong": 57 + }, + { + "budget": 0.56, + "threshold": -0.0070648193359375, + "operator": "ge", + "actual_fraction": 0.559254327563249, + "metric": 0.1696257903027952, + "rescued": 182, + "harmed": 1277, + "missed_rescue": 1016, + "avoided_harm": 832, + "selected_both_correct": 5186, + "selected_both_wrong": 75 + }, + { + "budget": 0.57, + "threshold": -0.007518768310546875, + "operator": "ge", + "actual_fraction": 0.572237017310253, + "metric": 0.17132213898552048, + "rescued": 211, + "harmed": 1300, + "missed_rescue": 987, + "avoided_harm": 809, + "selected_both_correct": 5276, + "selected_both_wrong": 89 + }, + { + "budget": 0.58, + "threshold": -0.00818634033203125, + "operator": "ge", + "actual_fraction": 0.5862183754993342, + "metric": 0.17290558782298915, + "rescued": 244, + "harmed": 1328, + "missed_rescue": 954, + "avoided_harm": 781, + "selected_both_correct": 5376, + "selected_both_wrong": 96 + }, + { + "budget": 0.59, + "threshold": -0.00878143310546875, + "operator": "gt", + "actual_fraction": 0.5998668442077231, + "metric": 0.1757330216164513, + "rescued": 281, + "harmed": 1358, + "missed_rescue": 917, + "avoided_harm": 751, + "selected_both_correct": 5460, + "selected_both_wrong": 109 + }, + { + "budget": 0.6, + "threshold": -0.00955963134765625, + "operator": "gt", + "actual_fraction": 0.6191744340878829, + "metric": 0.17918696387117494, + "rescued": 329, + "harmed": 1394, + "missed_rescue": 869, + "avoided_harm": 715, + "selected_both_correct": 5591, + "selected_both_wrong": 126 + }, + { + "budget": 0.61, + "threshold": -0.0099334716796875, + "operator": "ge", + "actual_fraction": 0.6264980026631158, + "metric": 0.18029020519999098, + "rescued": 347, + "harmed": 1407, + "missed_rescue": 851, + "avoided_harm": 702, + "selected_both_correct": 5643, + "selected_both_wrong": 131 + }, + { + "budget": 0.62, + "threshold": -0.010650634765625, + "operator": "gt", + "actual_fraction": 0.6384820239680427, + "metric": 0.18360411382005598, + "rescued": 376, + "harmed": 1424, + "missed_rescue": 822, + "avoided_harm": 685, + "selected_both_correct": 5729, + "selected_both_wrong": 143 + }, + { + "budget": 0.63, + "threshold": -0.01129150390625, + "operator": "ge", + "actual_fraction": 0.6461384820239681, + "metric": 0.18430872796712833, + "rescued": 393, + "harmed": 1439, + "missed_rescue": 805, + "avoided_harm": 670, + "selected_both_correct": 5780, + "selected_both_wrong": 152 + }, + { + "budget": 0.64, + "threshold": -0.011871337890625, + "operator": "ge", + "actual_fraction": 0.653462050599201, + "metric": 0.18413764925256976, + "rescued": 406, + "harmed": 1452, + "missed_rescue": 792, + "avoided_harm": 657, + "selected_both_correct": 5831, + "selected_both_wrong": 163 + }, + { + "budget": 0.65, + "threshold": -0.0125274658203125, + "operator": "gt", + "actual_fraction": 0.6597869507323568, + "metric": 0.18503793034066174, + "rescued": 422, + "harmed": 1464, + "missed_rescue": 776, + "avoided_harm": 645, + "selected_both_correct": 5873, + "selected_both_wrong": 169 + }, + { + "budget": 0.66, + "threshold": -0.012969970703125, + "operator": "gt", + "actual_fraction": 0.6661118508655126, + "metric": 0.186106537681911, + "rescued": 441, + "harmed": 1475, + "missed_rescue": 757, + "avoided_harm": 634, + "selected_both_correct": 5915, + "selected_both_wrong": 173 + }, + { + "budget": 0.67, + "threshold": -0.01363372802734375, + "operator": "ge", + "actual_fraction": 0.6734354194407457, + "metric": 0.1880916072476674, + "rescued": 461, + "harmed": 1484, + "missed_rescue": 737, + "avoided_harm": 625, + "selected_both_correct": 5966, + "selected_both_wrong": 181 + }, + { + "budget": 0.68, + "threshold": -0.01473236083984375, + "operator": "ge", + "actual_fraction": 0.6820905459387483, + "metric": 0.18987442927689613, + "rescued": 484, + "harmed": 1494, + "missed_rescue": 714, + "avoided_harm": 615, + "selected_both_correct": 6030, + "selected_both_wrong": 188 + }, + { + "budget": 0.69, + "threshold": -0.015716552734375, + "operator": "ge", + "actual_fraction": 0.6927430093209055, + "metric": 0.19160618326349385, + "rescued": 511, + "harmed": 1512, + "missed_rescue": 687, + "avoided_harm": 597, + "selected_both_correct": 6100, + "selected_both_wrong": 201 + }, + { + "budget": 0.7, + "threshold": -0.017578125, + "operator": "gt", + "actual_fraction": 0.7090545938748336, + "metric": 0.19565888484893704, + "rescued": 558, + "harmed": 1543, + "missed_rescue": 640, + "avoided_harm": 566, + "selected_both_correct": 6200, + "selected_both_wrong": 219 + }, + { + "budget": 0.71, + "threshold": -0.0189056396484375, + "operator": "ge", + "actual_fraction": 0.7170439414114514, + "metric": 0.1973992538335802, + "rescued": 581, + "harmed": 1558, + "missed_rescue": 617, + "avoided_harm": 551, + "selected_both_correct": 6250, + "selected_both_wrong": 227 + }, + { + "budget": 0.72, + "threshold": -0.019866943359375, + "operator": "ge", + "actual_fraction": 0.7243675099866844, + "metric": 0.1992894526144666, + "rescued": 603, + "harmed": 1570, + "missed_rescue": 595, + "avoided_harm": 539, + "selected_both_correct": 6292, + "selected_both_wrong": 239 + }, + { + "budget": 0.73, + "threshold": -0.020965576171875, + "operator": "ge", + "actual_fraction": 0.7330226364846871, + "metric": 0.2014755547124751, + "rescued": 624, + "harmed": 1582, + "missed_rescue": 574, + "avoided_harm": 527, + "selected_both_correct": 6355, + "selected_both_wrong": 247 + }, + { + "budget": 0.74, + "threshold": -0.021453857421875, + "operator": "ge", + "actual_fraction": 0.7363515312916112, + "metric": 0.20209035743657205, + "rescued": 634, + "harmed": 1592, + "missed_rescue": 564, + "avoided_harm": 517, + "selected_both_correct": 6374, + "selected_both_wrong": 248 + }, + { + "budget": 0.75, + "threshold": -0.0248870849609375, + "operator": "ge", + "actual_fraction": 0.7486684420772304, + "metric": 0.20546211362830763, + "rescued": 665, + "harmed": 1612, + "missed_rescue": 533, + "avoided_harm": 497, + "selected_both_correct": 6458, + "selected_both_wrong": 261 + }, + { + "budget": 0.76, + "threshold": -0.0269622802734375, + "operator": "ge", + "actual_fraction": 0.757323568575233, + "metric": 0.20773020664342898, + "rescued": 689, + "harmed": 1626, + "missed_rescue": 509, + "avoided_harm": 483, + "selected_both_correct": 6514, + "selected_both_wrong": 271 + }, + { + "budget": 0.77, + "threshold": -0.0282135009765625, + "operator": "ge", + "actual_fraction": 0.7639813581890812, + "metric": 0.2112193086129685, + "rescued": 707, + "harmed": 1633, + "missed_rescue": 491, + "avoided_harm": 476, + "selected_both_correct": 6558, + "selected_both_wrong": 282 + }, + { + "budget": 0.78, + "threshold": -0.0299835205078125, + "operator": "ge", + "actual_fraction": 0.7733022636484687, + "metric": 0.2156222025883268, + "rescued": 738, + "harmed": 1654, + "missed_rescue": 460, + "avoided_harm": 455, + "selected_both_correct": 6610, + "selected_both_wrong": 290 + }, + { + "budget": 0.79, + "threshold": -0.033966064453125, + "operator": "ge", + "actual_fraction": 0.784953395472703, + "metric": 0.21617077858796782, + "rescued": 763, + "harmed": 1672, + "missed_rescue": 435, + "avoided_harm": 437, + "selected_both_correct": 6690, + "selected_both_wrong": 307 + }, + { + "budget": 0.8, + "threshold": -0.037200927734375, + "operator": "ge", + "actual_fraction": 0.7939414114513982, + "metric": 0.21811777255990603, + "rescued": 788, + "harmed": 1693, + "missed_rescue": 410, + "avoided_harm": 416, + "selected_both_correct": 6742, + "selected_both_wrong": 317 + }, + { + "budget": 0.81, + "threshold": -0.04132080078125, + "operator": "ge", + "actual_fraction": 0.8039280958721704, + "metric": 0.22157947433765235, + "rescued": 814, + "harmed": 1705, + "missed_rescue": 384, + "avoided_harm": 404, + "selected_both_correct": 6809, + "selected_both_wrong": 332 + }, + { + "budget": 0.82, + "threshold": -0.0455322265625, + "operator": "ge", + "actual_fraction": 0.8122503328894807, + "metric": 0.22355438821966017, + "rescued": 835, + "harmed": 1722, + "missed_rescue": 363, + "avoided_harm": 387, + "selected_both_correct": 6857, + "selected_both_wrong": 346 + }, + { + "budget": 0.83, + "threshold": -0.051849365234375, + "operator": "ge", + "actual_fraction": 0.8215712383488681, + "metric": 0.22762279758152631, + "rescued": 863, + "harmed": 1740, + "missed_rescue": 335, + "avoided_harm": 369, + "selected_both_correct": 6911, + "selected_both_wrong": 358 + }, + { + "budget": 0.84, + "threshold": -0.06878662109375, + "operator": "ge", + "actual_fraction": 0.8388814913448736, + "metric": 0.23330777488772814, + "rescued": 909, + "harmed": 1767, + "missed_rescue": 289, + "avoided_harm": 342, + "selected_both_correct": 7021, + "selected_both_wrong": 383 + }, + { + "budget": 0.85, + "threshold": -0.08477783203125, + "operator": "ge", + "actual_fraction": 0.8535286284953395, + "metric": 0.23874788475407085, + "rescued": 947, + "harmed": 1794, + "missed_rescue": 251, + "avoided_harm": 315, + "selected_both_correct": 7110, + "selected_both_wrong": 405 + }, + { + "budget": 0.86, + "threshold": -0.09912109375, + "operator": "ge", + "actual_fraction": 0.8638482023968043, + "metric": 0.2419731256831217, + "rescued": 974, + "harmed": 1808, + "missed_rescue": 224, + "avoided_harm": 301, + "selected_both_correct": 7179, + "selected_both_wrong": 419 + }, + { + "budget": 0.87, + "threshold": -0.1298828125, + "operator": "ge", + "actual_fraction": 0.8745006657789614, + "metric": 0.2400850101807912, + "rescued": 996, + "harmed": 1832, + "missed_rescue": 202, + "avoided_harm": 277, + "selected_both_correct": 7252, + "selected_both_wrong": 428 + }, + { + "budget": 0.88, + "threshold": -0.1441650390625, + "operator": "ge", + "actual_fraction": 0.8764980026631158, + "metric": 0.24058030940047917, + "rescued": 1000, + "harmed": 1834, + "missed_rescue": 198, + "avoided_harm": 275, + "selected_both_correct": 7268, + "selected_both_wrong": 430 + }, + { + "budget": 0.89, + "threshold": -0.182373046875, + "operator": "ge", + "actual_fraction": 0.8841544607190412, + "metric": 0.24227931512045134, + "rescued": 1017, + "harmed": 1852, + "missed_rescue": 181, + "avoided_harm": 257, + "selected_both_correct": 7321, + "selected_both_wrong": 434 + }, + { + "budget": 0.9, + "threshold": -0.2049560546875, + "operator": "ge", + "actual_fraction": 0.8891478029294274, + "metric": 0.2414044779527529, + "rescued": 1030, + "harmed": 1861, + "missed_rescue": 168, + "avoided_harm": 248, + "selected_both_correct": 7354, + "selected_both_wrong": 439 + }, + { + "budget": 0.91, + "threshold": -0.25830078125, + "operator": "gt", + "actual_fraction": 0.8981358189081226, + "metric": 0.24325574507940786, + "rescued": 1055, + "harmed": 1885, + "missed_rescue": 143, + "avoided_harm": 224, + "selected_both_correct": 7403, + "selected_both_wrong": 449 + }, + { + "budget": 0.92, + "threshold": -0.32080078125, + "operator": "ge", + "actual_fraction": 0.9054593874833555, + "metric": 0.24437346394419443, + "rescued": 1069, + "harmed": 1898, + "missed_rescue": 129, + "avoided_harm": 211, + "selected_both_correct": 7455, + "selected_both_wrong": 458 + }, + { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9221038615179761, + "metric": 0.24877865873563798, + "rescued": 1105, + "harmed": 1927, + "missed_rescue": 93, + "avoided_harm": 182, + "selected_both_correct": 7575, + "selected_both_wrong": 473 + }, + { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9254327563249002, + "metric": 0.2523047077304935, + "rescued": 1113, + "harmed": 1932, + "missed_rescue": 85, + "avoided_harm": 177, + "selected_both_correct": 7597, + "selected_both_wrong": 478 + }, + { + "budget": 0.95, + "threshold": -0.611328125, + "operator": "ge", + "actual_fraction": 0.9364181091877497, + "metric": 0.2527194116218445, + "rescued": 1129, + "harmed": 1959, + "missed_rescue": 69, + "avoided_harm": 150, + "selected_both_correct": 7676, + "selected_both_wrong": 488 + }, + { + "budget": 0.96, + "threshold": -0.70068359375, + "operator": "ge", + "actual_fraction": 0.953728362183755, + "metric": 0.24014411561115867, + "rescued": 1148, + "harmed": 2005, + "missed_rescue": 50, + "avoided_harm": 104, + "selected_both_correct": 7804, + "selected_both_wrong": 503 + }, + { + "budget": 0.97, + "threshold": -0.75048828125, + "operator": "ge", + "actual_fraction": 0.96271637816245, + "metric": 0.2331497088327334, + "rescued": 1153, + "harmed": 2028, + "missed_rescue": 45, + "avoided_harm": 81, + "selected_both_correct": 7880, + "selected_both_wrong": 507 + }, + { + "budget": 0.98, + "threshold": -0.7841796875, + "operator": "ge", + "actual_fraction": 0.9700399467376831, + "metric": 0.23063278452496924, + "rescued": 1156, + "harmed": 2044, + "missed_rescue": 42, + "avoided_harm": 65, + "selected_both_correct": 7943, + "selected_both_wrong": 513 + }, + { + "budget": 0.99, + "threshold": -0.85107421875, + "operator": "ge", + "actual_fraction": 0.987017310252996, + "metric": 0.2242415410777232, + "rescued": 1174, + "harmed": 2087, + "missed_rescue": 24, + "avoided_harm": 22, + "selected_both_correct": 8072, + "selected_both_wrong": 527 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.22361115727328065, + "rescued": 1198, + "harmed": 2109, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 8176, + "selected_both_wrong": 533 + } + ] + }, + "learned": { + "selected_budget_index": 49, + "validation_area_0_25": 0.22914536623152232, + "validation_point": { + "budget": 0.49, + "threshold": -2.523495674133301, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.29007041551798585, + "rescued": 281, + "harmed": 227, + "missed_rescue": 5, + "avoided_harm": 291, + "selected_both_correct": 757, + "selected_both_wrong": 107 + }, + "test_point": { + "budget": 0.49, + "threshold": -2.523495674133301, + "operator": "ge", + "actual_fraction": 0.47203728362183756, + "metric": 0.2582183145054675, + "rescued": 1158, + "harmed": 850, + "missed_rescue": 40, + "avoided_harm": 1259, + "selected_both_correct": 3149, + "selected_both_wrong": 515 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.7074623107910156, + "operator": "ge", + "actual_fraction": 0.009985734664764621, + "metric": 0.18540459613418375, + "rescued": 10, + "harmed": 1, + "missed_rescue": 276, + "avoided_harm": 517, + "selected_both_correct": 10, + "selected_both_wrong": 7 + }, + { + "budget": 0.02, + "threshold": 1.5917527675628662, + "operator": "ge", + "actual_fraction": 0.019971469329529243, + "metric": 0.18841130478632961, + "rescued": 21, + "harmed": 3, + "missed_rescue": 265, + "avoided_harm": 515, + "selected_both_correct": 21, + "selected_both_wrong": 11 + }, + { + "budget": 0.03, + "threshold": 1.5071210861206055, + "operator": "ge", + "actual_fraction": 0.029957203994293864, + "metric": 0.1966616500929029, + "rescued": 34, + "harmed": 6, + "missed_rescue": 252, + "avoided_harm": 512, + "selected_both_correct": 31, + "selected_both_wrong": 13 + }, + { + "budget": 0.04, + "threshold": 1.4091997146606445, + "operator": "ge", + "actual_fraction": 0.039942938659058486, + "metric": 0.19887829130426626, + "rescued": 44, + "harmed": 9, + "missed_rescue": 242, + "avoided_harm": 509, + "selected_both_correct": 44, + "selected_both_wrong": 15 + }, + { + "budget": 0.05, + "threshold": 1.335000991821289, + "operator": "ge", + "actual_fraction": 0.04992867332382311, + "metric": 0.20307017543859646, + "rescued": 52, + "harmed": 13, + "missed_rescue": 234, + "avoided_harm": 505, + "selected_both_correct": 55, + "selected_both_wrong": 20 + }, + { + "budget": 0.06, + "threshold": 1.2808918952941895, + "operator": "ge", + "actual_fraction": 0.05991440798858773, + "metric": 0.20761625835004724, + "rescued": 64, + "harmed": 16, + "missed_rescue": 222, + "avoided_harm": 502, + "selected_both_correct": 67, + "selected_both_wrong": 21 + }, + { + "budget": 0.07, + "threshold": 1.08822762966156, + "operator": "ge", + "actual_fraction": 0.06990014265335236, + "metric": 0.2132973293889083, + "rescued": 77, + "harmed": 18, + "missed_rescue": 209, + "avoided_harm": 500, + "selected_both_correct": 79, + "selected_both_wrong": 22 + }, + { + "budget": 0.08, + "threshold": 0.9967296123504639, + "operator": "ge", + "actual_fraction": 0.07988587731811697, + "metric": 0.21958749784914022, + "rescued": 88, + "harmed": 19, + "missed_rescue": 198, + "avoided_harm": 499, + "selected_both_correct": 91, + "selected_both_wrong": 26 + }, + { + "budget": 0.09, + "threshold": 0.8405832648277283, + "operator": "ge", + "actual_fraction": 0.0898716119828816, + "metric": 0.22103428978428977, + "rescued": 99, + "harmed": 23, + "missed_rescue": 187, + "avoided_harm": 495, + "selected_both_correct": 102, + "selected_both_wrong": 28 + }, + { + "budget": 0.1, + "threshold": 0.5286899209022522, + "operator": "ge", + "actual_fraction": 0.09985734664764621, + "metric": 0.2249005660658424, + "rescued": 109, + "harmed": 26, + "missed_rescue": 177, + "avoided_harm": 492, + "selected_both_correct": 115, + "selected_both_wrong": 30 + }, + { + "budget": 0.11, + "threshold": 0.4132463335990906, + "operator": "ge", + "actual_fraction": 0.10984308131241084, + "metric": 0.22916056166056167, + "rescued": 115, + "harmed": 32, + "missed_rescue": 171, + "avoided_harm": 486, + "selected_both_correct": 130, + "selected_both_wrong": 31 + }, + { + "budget": 0.12, + "threshold": 0.37001603841781616, + "operator": "ge", + "actual_fraction": 0.11982881597717546, + "metric": 0.23141865473225776, + "rescued": 122, + "harmed": 37, + "missed_rescue": 164, + "avoided_harm": 481, + "selected_both_correct": 145, + "selected_both_wrong": 32 + }, + { + "budget": 0.13, + "threshold": 0.35031360387802124, + "operator": "ge", + "actual_fraction": 0.12981455064194009, + "metric": 0.23224298314927952, + "rescued": 126, + "harmed": 42, + "missed_rescue": 160, + "avoided_harm": 476, + "selected_both_correct": 163, + "selected_both_wrong": 33 + }, + { + "budget": 0.14, + "threshold": 0.32689201831817627, + "operator": "ge", + "actual_fraction": 0.1398002853067047, + "metric": 0.23533898753711674, + "rescued": 131, + "harmed": 46, + "missed_rescue": 155, + "avoided_harm": 472, + "selected_both_correct": 180, + "selected_both_wrong": 35 + }, + { + "budget": 0.15, + "threshold": 0.29726189374923706, + "operator": "ge", + "actual_fraction": 0.14978601997146934, + "metric": 0.24932114884173076, + "rescued": 139, + "harmed": 51, + "missed_rescue": 147, + "avoided_harm": 467, + "selected_both_correct": 195, + "selected_both_wrong": 35 + }, + { + "budget": 0.16, + "threshold": 0.27329546213150024, + "operator": "ge", + "actual_fraction": 0.15977175463623394, + "metric": 0.2502195362606866, + "rescued": 145, + "harmed": 56, + "missed_rescue": 141, + "avoided_harm": 462, + "selected_both_correct": 211, + "selected_both_wrong": 36 + }, + { + "budget": 0.17, + "threshold": 0.23806273937225342, + "operator": "ge", + "actual_fraction": 0.16975748930099857, + "metric": 0.2532928388746803, + "rescued": 150, + "harmed": 60, + "missed_rescue": 136, + "avoided_harm": 458, + "selected_both_correct": 228, + "selected_both_wrong": 38 + }, + { + "budget": 0.18, + "threshold": 0.21703343093395233, + "operator": "ge", + "actual_fraction": 0.1797432239657632, + "metric": 0.2548397364593138, + "rescued": 156, + "harmed": 64, + "missed_rescue": 130, + "avoided_harm": 454, + "selected_both_correct": 245, + "selected_both_wrong": 39 + }, + { + "budget": 0.19, + "threshold": 0.17793023586273193, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.2546125997568839, + "rescued": 158, + "harmed": 69, + "missed_rescue": 128, + "avoided_harm": 449, + "selected_both_correct": 262, + "selected_both_wrong": 43 + }, + { + "budget": 0.2, + "threshold": 0.16312265396118164, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.25527351436442347, + "rescued": 164, + "harmed": 74, + "missed_rescue": 122, + "avoided_harm": 444, + "selected_both_correct": 278, + "selected_both_wrong": 44 + }, + { + "budget": 0.21, + "threshold": 0.14629703760147095, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.25962415341845474, + "rescued": 170, + "harmed": 79, + "missed_rescue": 116, + "avoided_harm": 439, + "selected_both_correct": 294, + "selected_both_wrong": 45 + }, + { + "budget": 0.22, + "threshold": 0.12397575378417969, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.2582672584170057, + "rescued": 175, + "harmed": 84, + "missed_rescue": 111, + "avoided_harm": 434, + "selected_both_correct": 311, + "selected_both_wrong": 46 + }, + { + "budget": 0.23, + "threshold": 0.09866081178188324, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.2600712178182914, + "rescued": 179, + "harmed": 86, + "missed_rescue": 107, + "avoided_harm": 432, + "selected_both_correct": 330, + "selected_both_wrong": 49 + }, + { + "budget": 0.24, + "threshold": 0.047809898853302, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.25954693897277625, + "rescued": 184, + "harmed": 89, + "missed_rescue": 102, + "avoided_harm": 429, + "selected_both_correct": 348, + "selected_both_wrong": 51 + }, + { + "budget": 0.25, + "threshold": 0.017505675554275513, + "operator": "ge", + "actual_fraction": 0.24964336661911554, + "metric": 0.24378213120437914, + "rescued": 188, + "harmed": 97, + "missed_rescue": 98, + "avoided_harm": 421, + "selected_both_correct": 362, + "selected_both_wrong": 53 + }, + { + "budget": 0.26, + "threshold": -0.030715346336364746, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.24422495312906273, + "rescued": 192, + "harmed": 103, + "missed_rescue": 94, + "avoided_harm": 415, + "selected_both_correct": 378, + "selected_both_wrong": 55 + }, + { + "budget": 0.27, + "threshold": -0.09052437543869019, + "operator": "ge", + "actual_fraction": 0.26961483594864477, + "metric": 0.24499903586891705, + "rescued": 197, + "harmed": 109, + "missed_rescue": 89, + "avoided_harm": 409, + "selected_both_correct": 393, + "selected_both_wrong": 57 + }, + { + "budget": 0.28, + "threshold": -0.138982892036438, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.250255339891454, + "rescued": 201, + "harmed": 115, + "missed_rescue": 85, + "avoided_harm": 403, + "selected_both_correct": 409, + "selected_both_wrong": 59 + }, + { + "budget": 0.29, + "threshold": -0.16997069120407104, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.2515088180537899, + "rescued": 206, + "harmed": 123, + "missed_rescue": 80, + "avoided_harm": 395, + "selected_both_correct": 423, + "selected_both_wrong": 60 + }, + { + "budget": 0.3, + "threshold": -0.20665131509304047, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.25345413401110894, + "rescued": 210, + "harmed": 131, + "missed_rescue": 76, + "avoided_harm": 387, + "selected_both_correct": 437, + "selected_both_wrong": 62 + }, + { + "budget": 0.31, + "threshold": -0.2492215782403946, + "operator": "ge", + "actual_fraction": 0.3095577746077033, + "metric": 0.2556197710521957, + "rescued": 215, + "harmed": 136, + "missed_rescue": 71, + "avoided_harm": 382, + "selected_both_correct": 453, + "selected_both_wrong": 64 + }, + { + "budget": 0.32, + "threshold": -0.2695646286010742, + "operator": "ge", + "actual_fraction": 0.3195435092724679, + "metric": 0.2567479487769343, + "rescued": 220, + "harmed": 142, + "missed_rescue": 66, + "avoided_harm": 376, + "selected_both_correct": 469, + "selected_both_wrong": 65 + }, + { + "budget": 0.33, + "threshold": -0.2922053933143616, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.2572916666666667, + "rescued": 225, + "harmed": 150, + "missed_rescue": 61, + "avoided_harm": 368, + "selected_both_correct": 482, + "selected_both_wrong": 67 + }, + { + "budget": 0.34, + "threshold": -0.32532864809036255, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.2565012845475888, + "rescued": 228, + "harmed": 156, + "missed_rescue": 58, + "avoided_harm": 362, + "selected_both_correct": 498, + "selected_both_wrong": 70 + }, + { + "budget": 0.35, + "threshold": -0.33954745531082153, + "operator": "ge", + "actual_fraction": 0.34950071326676174, + "metric": 0.2657392749791891, + "rescued": 234, + "harmed": 163, + "missed_rescue": 52, + "avoided_harm": 355, + "selected_both_correct": 512, + "selected_both_wrong": 71 + }, + { + "budget": 0.36, + "threshold": -0.3620870113372803, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.27242891417825765, + "rescued": 239, + "harmed": 168, + "missed_rescue": 47, + "avoided_harm": 350, + "selected_both_correct": 528, + "selected_both_wrong": 73 + }, + { + "budget": 0.37, + "threshold": -0.4306718707084656, + "operator": "ge", + "actual_fraction": 0.369472182596291, + "metric": 0.2744237734705873, + "rescued": 244, + "harmed": 173, + "missed_rescue": 42, + "avoided_harm": 345, + "selected_both_correct": 545, + "selected_both_wrong": 74 + }, + { + "budget": 0.38, + "threshold": -0.4907958507537842, + "operator": "ge", + "actual_fraction": 0.37945791726105566, + "metric": 0.27225965684570924, + "rescued": 247, + "harmed": 176, + "missed_rescue": 39, + "avoided_harm": 342, + "selected_both_correct": 564, + "selected_both_wrong": 77 + }, + { + "budget": 0.39, + "threshold": -0.5423566699028015, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.28304295730582607, + "rescued": 252, + "harmed": 180, + "missed_rescue": 34, + "avoided_harm": 338, + "selected_both_correct": 580, + "selected_both_wrong": 80 + }, + { + "budget": 0.4, + "threshold": -0.6501604318618774, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.2806420813939611, + "rescued": 254, + "harmed": 185, + "missed_rescue": 32, + "avoided_harm": 333, + "selected_both_correct": 598, + "selected_both_wrong": 83 + }, + { + "budget": 0.41, + "threshold": -0.7341160178184509, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.284008025391733, + "rescued": 258, + "harmed": 193, + "missed_rescue": 28, + "avoided_harm": 325, + "selected_both_correct": 610, + "selected_both_wrong": 87 + }, + { + "budget": 0.42, + "threshold": -0.8236439824104309, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.27884208539197114, + "rescued": 262, + "harmed": 199, + "missed_rescue": 24, + "avoided_harm": 319, + "selected_both_correct": 624, + "selected_both_wrong": 91 + }, + { + "budget": 0.43, + "threshold": -0.9715824723243713, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.28027100878456984, + "rescued": 265, + "harmed": 202, + "missed_rescue": 21, + "avoided_harm": 316, + "selected_both_correct": 644, + "selected_both_wrong": 93 + }, + { + "budget": 0.44, + "threshold": -1.0671179294586182, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.28408657253448183, + "rescued": 267, + "harmed": 203, + "missed_rescue": 19, + "avoided_harm": 315, + "selected_both_correct": 664, + "selected_both_wrong": 98 + }, + { + "budget": 0.45, + "threshold": -1.2702796459197998, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.28705454715376755, + "rescued": 271, + "harmed": 205, + "missed_rescue": 15, + "avoided_harm": 313, + "selected_both_correct": 682, + "selected_both_wrong": 102 + }, + { + "budget": 0.46, + "threshold": -1.5780962705612183, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.28722189653104074, + "rescued": 275, + "harmed": 210, + "missed_rescue": 11, + "avoided_harm": 308, + "selected_both_correct": 700, + "selected_both_wrong": 103 + }, + { + "budget": 0.47, + "threshold": -2.1831114292144775, + "operator": "ge", + "actual_fraction": 0.4693295292439372, + "metric": 0.28351130477013453, + "rescued": 276, + "harmed": 217, + "missed_rescue": 10, + "avoided_harm": 301, + "selected_both_correct": 720, + "selected_both_wrong": 103 + }, + { + "budget": 0.48, + "threshold": -2.363466501235962, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.28473722809103874, + "rescued": 278, + "harmed": 221, + "missed_rescue": 8, + "avoided_harm": 297, + "selected_both_correct": 740, + "selected_both_wrong": 105 + }, + { + "budget": 0.49, + "threshold": -2.523495674133301, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.29007041551798585, + "rescued": 281, + "harmed": 227, + "missed_rescue": 5, + "avoided_harm": 291, + "selected_both_correct": 757, + "selected_both_wrong": 107 + }, + { + "budget": 0.5, + "threshold": -2.882524013519287, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.2882439482351161, + "rescued": 282, + "harmed": 231, + "missed_rescue": 4, + "avoided_harm": 287, + "selected_both_correct": 780, + "selected_both_wrong": 107 + }, + { + "budget": 0.51, + "threshold": -3.2156200408935547, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.28086742562597294, + "rescued": 283, + "harmed": 242, + "missed_rescue": 3, + "avoided_harm": 276, + "selected_both_correct": 796, + "selected_both_wrong": 107 + }, + { + "budget": 0.52, + "threshold": -4.331794738769531, + "operator": "ge", + "actual_fraction": 0.5192582025677603, + "metric": 0.27742622015067836, + "rescued": 283, + "harmed": 249, + "missed_rescue": 3, + "avoided_harm": 269, + "selected_both_correct": 817, + "selected_both_wrong": 107 + }, + { + "budget": 0.53, + "threshold": -4.379722595214844, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.2758119707358199, + "rescued": 283, + "harmed": 254, + "missed_rescue": 3, + "avoided_harm": 264, + "selected_both_correct": 840, + "selected_both_wrong": 107 + }, + { + "budget": 0.54, + "threshold": -4.431995391845703, + "operator": "ge", + "actual_fraction": 0.5392296718972895, + "metric": 0.2698097256920786, + "rescued": 283, + "harmed": 262, + "missed_rescue": 3, + "avoided_harm": 256, + "selected_both_correct": 860, + "selected_both_wrong": 107 + }, + { + "budget": 0.55, + "threshold": -4.467194557189941, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.2689410722197607, + "rescued": 283, + "harmed": 268, + "missed_rescue": 3, + "avoided_harm": 250, + "selected_both_correct": 882, + "selected_both_wrong": 107 + }, + { + "budget": 0.56, + "threshold": -4.493173122406006, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.27060957663222585, + "rescued": 284, + "harmed": 274, + "missed_rescue": 2, + "avoided_harm": 244, + "selected_both_correct": 903, + "selected_both_wrong": 107 + }, + { + "budget": 0.57, + "threshold": -4.505556106567383, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.2694706660477131, + "rescued": 284, + "harmed": 282, + "missed_rescue": 2, + "avoided_harm": 236, + "selected_both_correct": 923, + "selected_both_wrong": 107 + }, + { + "budget": 0.58, + "threshold": -4.531030654907227, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.2678380311075578, + "rescued": 284, + "harmed": 288, + "missed_rescue": 2, + "avoided_harm": 230, + "selected_both_correct": 945, + "selected_both_wrong": 107 + }, + { + "budget": 0.59, + "threshold": -4.553508758544922, + "operator": "ge", + "actual_fraction": 0.5891583452211127, + "metric": 0.26717829717606256, + "rescued": 284, + "harmed": 293, + "missed_rescue": 2, + "avoided_harm": 225, + "selected_both_correct": 968, + "selected_both_wrong": 107 + }, + { + "budget": 0.6, + "threshold": -4.568713188171387, + "operator": "ge", + "actual_fraction": 0.5991440798858774, + "metric": 0.26629189984221124, + "rescued": 284, + "harmed": 300, + "missed_rescue": 2, + "avoided_harm": 218, + "selected_both_correct": 989, + "selected_both_wrong": 107 + }, + { + "budget": 0.61, + "threshold": -4.596630096435547, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.2606395046261513, + "rescued": 284, + "harmed": 305, + "missed_rescue": 2, + "avoided_harm": 213, + "selected_both_correct": 1012, + "selected_both_wrong": 107 + }, + { + "budget": 0.62, + "threshold": -4.6132402420043945, + "operator": "ge", + "actual_fraction": 0.6191155492154066, + "metric": 0.2601938670204924, + "rescued": 284, + "harmed": 309, + "missed_rescue": 2, + "avoided_harm": 209, + "selected_both_correct": 1036, + "selected_both_wrong": 107 + }, + { + "budget": 0.63, + "threshold": -4.62782621383667, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.25877466157547113, + "rescued": 284, + "harmed": 314, + "missed_rescue": 2, + "avoided_harm": 204, + "selected_both_correct": 1059, + "selected_both_wrong": 107 + }, + { + "budget": 0.64, + "threshold": -4.642134666442871, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.2573560110319158, + "rescued": 284, + "harmed": 319, + "missed_rescue": 2, + "avoided_harm": 199, + "selected_both_correct": 1082, + "selected_both_wrong": 107 + }, + { + "budget": 0.65, + "threshold": -4.669188976287842, + "operator": "ge", + "actual_fraction": 0.6490727532097005, + "metric": 0.2589982538880948, + "rescued": 285, + "harmed": 324, + "missed_rescue": 1, + "avoided_harm": 194, + "selected_both_correct": 1104, + "selected_both_wrong": 107 + }, + { + "budget": 0.66, + "threshold": -4.687463760375977, + "operator": "ge", + "actual_fraction": 0.6590584878744651, + "metric": 0.2584884831125433, + "rescued": 285, + "harmed": 329, + "missed_rescue": 1, + "avoided_harm": 189, + "selected_both_correct": 1127, + "selected_both_wrong": 107 + }, + { + "budget": 0.67, + "threshold": -4.702577590942383, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.258000065855731, + "rescued": 285, + "harmed": 334, + "missed_rescue": 1, + "avoided_harm": 184, + "selected_both_correct": 1150, + "selected_both_wrong": 107 + }, + { + "budget": 0.68, + "threshold": -4.743311405181885, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.253502517434406, + "rescued": 285, + "harmed": 340, + "missed_rescue": 1, + "avoided_harm": 178, + "selected_both_correct": 1172, + "selected_both_wrong": 107 + }, + { + "budget": 0.69, + "threshold": -4.790846824645996, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.25296695013970627, + "rescued": 285, + "harmed": 345, + "missed_rescue": 1, + "avoided_harm": 173, + "selected_both_correct": 1195, + "selected_both_wrong": 107 + }, + { + "budget": 0.7, + "threshold": -4.810942649841309, + "operator": "ge", + "actual_fraction": 0.6990014265335235, + "metric": 0.2510539714065037, + "rescued": 286, + "harmed": 351, + "missed_rescue": 0, + "avoided_harm": 167, + "selected_both_correct": 1216, + "selected_both_wrong": 107 + }, + { + "budget": 0.71, + "threshold": -4.89448356628418, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.25026969597907817, + "rescued": 286, + "harmed": 358, + "missed_rescue": 0, + "avoided_harm": 160, + "selected_both_correct": 1237, + "selected_both_wrong": 107 + }, + { + "budget": 0.72, + "threshold": -4.957235336303711, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.24885040774781703, + "rescued": 286, + "harmed": 364, + "missed_rescue": 0, + "avoided_harm": 154, + "selected_both_correct": 1259, + "selected_both_wrong": 107 + }, + { + "budget": 0.73, + "threshold": -4.993195533752441, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.2472982541131752, + "rescued": 286, + "harmed": 371, + "missed_rescue": 0, + "avoided_harm": 147, + "selected_both_correct": 1280, + "selected_both_wrong": 107 + }, + { + "budget": 0.74, + "threshold": -5.04953145980835, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.24347148993805237, + "rescued": 286, + "harmed": 379, + "missed_rescue": 0, + "avoided_harm": 139, + "selected_both_correct": 1300, + "selected_both_wrong": 107 + }, + { + "budget": 0.75, + "threshold": -5.097166538238525, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.24289537439861464, + "rescued": 286, + "harmed": 386, + "missed_rescue": 0, + "avoided_harm": 132, + "selected_both_correct": 1321, + "selected_both_wrong": 107 + }, + { + "budget": 0.76, + "threshold": -5.135637283325195, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.23914388179126791, + "rescued": 286, + "harmed": 392, + "missed_rescue": 0, + "avoided_harm": 126, + "selected_both_correct": 1343, + "selected_both_wrong": 107 + }, + { + "budget": 0.77, + "threshold": -5.158949375152588, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.23785596042417734, + "rescued": 286, + "harmed": 402, + "missed_rescue": 0, + "avoided_harm": 116, + "selected_both_correct": 1360, + "selected_both_wrong": 108 + }, + { + "budget": 0.78, + "threshold": -5.1902618408203125, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.23590450445772781, + "rescued": 286, + "harmed": 407, + "missed_rescue": 0, + "avoided_harm": 111, + "selected_both_correct": 1383, + "selected_both_wrong": 108 + }, + { + "budget": 0.79, + "threshold": -5.255657196044922, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.23234022523621906, + "rescued": 286, + "harmed": 414, + "missed_rescue": 0, + "avoided_harm": 104, + "selected_both_correct": 1404, + "selected_both_wrong": 108 + }, + { + "budget": 0.8, + "threshold": -5.313392162322998, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.22827302995553944, + "rescued": 286, + "harmed": 422, + "missed_rescue": 0, + "avoided_harm": 96, + "selected_both_correct": 1424, + "selected_both_wrong": 108 + }, + { + "budget": 0.81, + "threshold": -5.414909362792969, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.22305746257574982, + "rescued": 286, + "harmed": 429, + "missed_rescue": 0, + "avoided_harm": 89, + "selected_both_correct": 1444, + "selected_both_wrong": 109 + }, + { + "budget": 0.82, + "threshold": -12.358579635620117, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.2227162303672329, + "rescued": 286, + "harmed": 434, + "missed_rescue": 0, + "avoided_harm": 84, + "selected_both_correct": 1467, + "selected_both_wrong": 109 + }, + { + "budget": 0.83, + "threshold": -12.387406349182129, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.22155979751115404, + "rescued": 286, + "harmed": 440, + "missed_rescue": 0, + "avoided_harm": 78, + "selected_both_correct": 1489, + "selected_both_wrong": 109 + }, + { + "budget": 0.84, + "threshold": -12.407716751098633, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.22053713884332193, + "rescued": 286, + "harmed": 444, + "missed_rescue": 0, + "avoided_harm": 74, + "selected_both_correct": 1513, + "selected_both_wrong": 109 + }, + { + "budget": 0.85, + "threshold": -12.423635482788086, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.2203033001616794, + "rescued": 286, + "harmed": 447, + "missed_rescue": 0, + "avoided_harm": 71, + "selected_both_correct": 1538, + "selected_both_wrong": 109 + }, + { + "budget": 0.86, + "threshold": -12.440216064453125, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.21852988259083223, + "rescued": 286, + "harmed": 452, + "missed_rescue": 0, + "avoided_harm": 66, + "selected_both_correct": 1561, + "selected_both_wrong": 109 + }, + { + "budget": 0.87, + "threshold": -12.449625015258789, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.21802213861759343, + "rescued": 286, + "harmed": 459, + "missed_rescue": 0, + "avoided_harm": 59, + "selected_both_correct": 1582, + "selected_both_wrong": 109 + }, + { + "budget": 0.88, + "threshold": -12.47104263305664, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.21761197152182887, + "rescued": 286, + "harmed": 464, + "missed_rescue": 0, + "avoided_harm": 54, + "selected_both_correct": 1605, + "selected_both_wrong": 109 + }, + { + "budget": 0.89, + "threshold": -12.487972259521484, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.21744835003287333, + "rescued": 286, + "harmed": 467, + "missed_rescue": 0, + "avoided_harm": 51, + "selected_both_correct": 1630, + "selected_both_wrong": 109 + }, + { + "budget": 0.9, + "threshold": -12.537336349487305, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.2149929340118669, + "rescued": 286, + "harmed": 473, + "missed_rescue": 0, + "avoided_harm": 45, + "selected_both_correct": 1652, + "selected_both_wrong": 109 + }, + { + "budget": 0.91, + "threshold": -12.567386627197266, + "operator": "ge", + "actual_fraction": 0.9087018544935807, + "metric": 0.21398158003305917, + "rescued": 286, + "harmed": 479, + "missed_rescue": 0, + "avoided_harm": 39, + "selected_both_correct": 1674, + "selected_both_wrong": 109 + }, + { + "budget": 0.92, + "threshold": -12.593729972839355, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.21236498430784465, + "rescued": 286, + "harmed": 483, + "missed_rescue": 0, + "avoided_harm": 35, + "selected_both_correct": 1698, + "selected_both_wrong": 109 + }, + { + "budget": 0.93, + "threshold": -12.62214469909668, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.21209409289366346, + "rescued": 286, + "harmed": 488, + "missed_rescue": 0, + "avoided_harm": 30, + "selected_both_correct": 1721, + "selected_both_wrong": 109 + }, + { + "budget": 0.94, + "threshold": -12.674238204956055, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.21179908167902023, + "rescued": 286, + "harmed": 493, + "missed_rescue": 0, + "avoided_harm": 25, + "selected_both_correct": 1744, + "selected_both_wrong": 109 + }, + { + "budget": 0.95, + "threshold": -12.761373519897461, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.21151014682189503, + "rescued": 286, + "harmed": 498, + "missed_rescue": 0, + "avoided_harm": 20, + "selected_both_correct": 1767, + "selected_both_wrong": 109 + }, + { + "budget": 0.96, + "threshold": -12.814491271972656, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.20925535326760145, + "rescued": 286, + "harmed": 503, + "missed_rescue": 0, + "avoided_harm": 15, + "selected_both_correct": 1790, + "selected_both_wrong": 109 + }, + { + "budget": 0.97, + "threshold": -12.931108474731445, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.20909356864186918, + "rescued": 286, + "harmed": 506, + "missed_rescue": 0, + "avoided_harm": 12, + "selected_both_correct": 1815, + "selected_both_wrong": 109 + }, + { + "budget": 0.98, + "threshold": -12.97988510131836, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.2089338953482874, + "rescued": 286, + "harmed": 509, + "missed_rescue": 0, + "avoided_harm": 9, + "selected_both_correct": 1840, + "selected_both_wrong": 109 + }, + { + "budget": 0.99, + "threshold": -13.074522018432617, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.20870333412712117, + "rescued": 286, + "harmed": 513, + "missed_rescue": 0, + "avoided_harm": 5, + "selected_both_correct": 1864, + "selected_both_wrong": 109 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.20666016863629594, + "rescued": 286, + "harmed": 518, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 1891, + "selected_both_wrong": 109 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.7074623107910156, + "operator": "ge", + "actual_fraction": 0.009986684420772303, + "metric": 0.18303777954083086, + "rescued": 53, + "harmed": 5, + "missed_rescue": 1145, + "avoided_harm": 2104, + "selected_both_correct": 33, + "selected_both_wrong": 29 + }, + { + "budget": 0.02, + "threshold": 1.5917527675628662, + "operator": "ge", + "actual_fraction": 0.017976031957390146, + "metric": 0.1851615087112737, + "rescued": 90, + "harmed": 14, + "missed_rescue": 1108, + "avoided_harm": 2095, + "selected_both_correct": 70, + "selected_both_wrong": 42 + }, + { + "budget": 0.03, + "threshold": 1.5071210861206055, + "operator": "ge", + "actual_fraction": 0.02729693741677763, + "metric": 0.18897863526398181, + "rescued": 135, + "harmed": 23, + "missed_rescue": 1063, + "avoided_harm": 2086, + "selected_both_correct": 114, + "selected_both_wrong": 56 + }, + { + "budget": 0.04, + "threshold": 1.4091997146606445, + "operator": "ge", + "actual_fraction": 0.039280958721704395, + "metric": 0.19594486238007697, + "rescued": 184, + "harmed": 38, + "missed_rescue": 1014, + "avoided_harm": 2071, + "selected_both_correct": 173, + "selected_both_wrong": 77 + }, + { + "budget": 0.05, + "threshold": 1.335000991821289, + "operator": "ge", + "actual_fraction": 0.04760319573901465, + "metric": 0.19703929583914914, + "rescued": 214, + "harmed": 48, + "missed_rescue": 984, + "avoided_harm": 2061, + "selected_both_correct": 220, + "selected_both_wrong": 90 + }, + { + "budget": 0.06, + "threshold": 1.2808918952941895, + "operator": "ge", + "actual_fraction": 0.05226364846870839, + "metric": 0.19836197779309986, + "rescued": 231, + "harmed": 51, + "missed_rescue": 967, + "avoided_harm": 2058, + "selected_both_correct": 245, + "selected_both_wrong": 101 + }, + { + "budget": 0.07, + "threshold": 1.08822762966156, + "operator": "ge", + "actual_fraction": 0.06624500665778961, + "metric": 0.20089171237387268, + "rescued": 274, + "harmed": 68, + "missed_rescue": 924, + "avoided_harm": 2041, + "selected_both_correct": 317, + "selected_both_wrong": 137 + }, + { + "budget": 0.08, + "threshold": 0.9967296123504639, + "operator": "ge", + "actual_fraction": 0.07390146471371505, + "metric": 0.20233999230766453, + "rescued": 305, + "harmed": 83, + "missed_rescue": 893, + "avoided_harm": 2026, + "selected_both_correct": 349, + "selected_both_wrong": 151 + }, + { + "budget": 0.09, + "threshold": 0.8405832648277283, + "operator": "ge", + "actual_fraction": 0.08621837549933423, + "metric": 0.20618791921914176, + "rescued": 352, + "harmed": 99, + "missed_rescue": 846, + "avoided_harm": 2010, + "selected_both_correct": 413, + "selected_both_wrong": 172 + }, + { + "budget": 0.1, + "threshold": 0.5286899209022522, + "operator": "ge", + "actual_fraction": 0.1085219707057257, + "metric": 0.21012410016243807, + "rescued": 426, + "harmed": 134, + "missed_rescue": 772, + "avoided_harm": 1975, + "selected_both_correct": 530, + "selected_both_wrong": 214 + }, + { + "budget": 0.11, + "threshold": 0.4132463335990906, + "operator": "ge", + "actual_fraction": 0.11984021304926765, + "metric": 0.21439476314025255, + "rescued": 460, + "harmed": 152, + "missed_rescue": 738, + "avoided_harm": 1957, + "selected_both_correct": 603, + "selected_both_wrong": 225 + }, + { + "budget": 0.12, + "threshold": 0.37001603841781616, + "operator": "ge", + "actual_fraction": 0.1278295605858855, + "metric": 0.2135891737248197, + "rescued": 482, + "harmed": 171, + "missed_rescue": 716, + "avoided_harm": 1938, + "selected_both_correct": 654, + "selected_both_wrong": 229 + }, + { + "budget": 0.13, + "threshold": 0.35031360387802124, + "operator": "ge", + "actual_fraction": 0.13648468708388814, + "metric": 0.2153575533947741, + "rescued": 502, + "harmed": 190, + "missed_rescue": 696, + "avoided_harm": 1919, + "selected_both_correct": 714, + "selected_both_wrong": 234 + }, + { + "budget": 0.14, + "threshold": 0.32689201831817627, + "operator": "ge", + "actual_fraction": 0.14613848202396804, + "metric": 0.223648825136357, + "rescued": 532, + "harmed": 212, + "missed_rescue": 666, + "avoided_harm": 1897, + "selected_both_correct": 776, + "selected_both_wrong": 236 + }, + { + "budget": 0.15, + "threshold": 0.29726189374923706, + "operator": "ge", + "actual_fraction": 0.1581225033288948, + "metric": 0.22656283237486058, + "rescued": 564, + "harmed": 231, + "missed_rescue": 634, + "avoided_harm": 1878, + "selected_both_correct": 861, + "selected_both_wrong": 244 + }, + { + "budget": 0.16, + "threshold": 0.27329546213150024, + "operator": "ge", + "actual_fraction": 0.16577896138482023, + "metric": 0.2285261963739467, + "rescued": 582, + "harmed": 247, + "missed_rescue": 616, + "avoided_harm": 1862, + "selected_both_correct": 913, + "selected_both_wrong": 250 + }, + { + "budget": 0.17, + "threshold": 0.23806273937225342, + "operator": "ge", + "actual_fraction": 0.1780958721704394, + "metric": 0.23156269579554606, + "rescued": 615, + "harmed": 267, + "missed_rescue": 583, + "avoided_harm": 1842, + "selected_both_correct": 1004, + "selected_both_wrong": 254 + }, + { + "budget": 0.18, + "threshold": 0.21703343093395233, + "operator": "ge", + "actual_fraction": 0.18841544607190414, + "metric": 0.23358838086817144, + "rescued": 642, + "harmed": 291, + "missed_rescue": 556, + "avoided_harm": 1818, + "selected_both_correct": 1077, + "selected_both_wrong": 254 + }, + { + "budget": 0.19, + "threshold": 0.17793023586273193, + "operator": "ge", + "actual_fraction": 0.2040612516644474, + "metric": 0.2348735775248642, + "rescued": 678, + "harmed": 326, + "missed_rescue": 520, + "avoided_harm": 1783, + "selected_both_correct": 1185, + "selected_both_wrong": 263 + }, + { + "budget": 0.2, + "threshold": 0.16312265396118164, + "operator": "ge", + "actual_fraction": 0.20872170439414114, + "metric": 0.2397534542325863, + "rescued": 689, + "harmed": 335, + "missed_rescue": 509, + "avoided_harm": 1774, + "selected_both_correct": 1219, + "selected_both_wrong": 265 + }, + { + "budget": 0.21, + "threshold": 0.14629703760147095, + "operator": "ge", + "actual_fraction": 0.21471371504660453, + "metric": 0.23955945106085613, + "rescued": 701, + "harmed": 346, + "missed_rescue": 497, + "avoided_harm": 1763, + "selected_both_correct": 1262, + "selected_both_wrong": 271 + }, + { + "budget": 0.22, + "threshold": 0.12397575378417969, + "operator": "ge", + "actual_fraction": 0.22636484687083888, + "metric": 0.23920328295328294, + "rescued": 727, + "harmed": 375, + "missed_rescue": 471, + "avoided_harm": 1734, + "selected_both_correct": 1339, + "selected_both_wrong": 279 + }, + { + "budget": 0.23, + "threshold": 0.09866081178188324, + "operator": "ge", + "actual_fraction": 0.23601864181091878, + "metric": 0.24001599302268145, + "rescued": 750, + "harmed": 396, + "missed_rescue": 448, + "avoided_harm": 1713, + "selected_both_correct": 1405, + "selected_both_wrong": 285 + }, + { + "budget": 0.24, + "threshold": 0.047809898853302, + "operator": "ge", + "actual_fraction": 0.2513315579227696, + "metric": 0.2437651800474398, + "rescued": 781, + "harmed": 427, + "missed_rescue": 417, + "avoided_harm": 1682, + "selected_both_correct": 1508, + "selected_both_wrong": 304 + }, + { + "budget": 0.25, + "threshold": 0.017505675554275513, + "operator": "ge", + "actual_fraction": 0.25865512649800265, + "metric": 0.2450836279564324, + "rescued": 798, + "harmed": 444, + "missed_rescue": 400, + "avoided_harm": 1665, + "selected_both_correct": 1556, + "selected_both_wrong": 310 + }, + { + "budget": 0.26, + "threshold": -0.030715346336364746, + "operator": "ge", + "actual_fraction": 0.26964047936085217, + "metric": 0.245209185258466, + "rescued": 821, + "harmed": 468, + "missed_rescue": 377, + "avoided_harm": 1641, + "selected_both_correct": 1633, + "selected_both_wrong": 318 + }, + { + "budget": 0.27, + "threshold": -0.09052437543869019, + "operator": "ge", + "actual_fraction": 0.2829560585885486, + "metric": 0.2459726849369736, + "rescued": 849, + "harmed": 493, + "missed_rescue": 349, + "avoided_harm": 1616, + "selected_both_correct": 1728, + "selected_both_wrong": 330 + }, + { + "budget": 0.28, + "threshold": -0.138982892036438, + "operator": "ge", + "actual_fraction": 0.2966045272969374, + "metric": 0.24456905524406827, + "rescued": 882, + "harmed": 522, + "missed_rescue": 316, + "avoided_harm": 1587, + "selected_both_correct": 1827, + "selected_both_wrong": 333 + }, + { + "budget": 0.29, + "threshold": -0.16997069120407104, + "operator": "ge", + "actual_fraction": 0.303595206391478, + "metric": 0.24690038334564665, + "rescued": 899, + "harmed": 536, + "missed_rescue": 299, + "avoided_harm": 1573, + "selected_both_correct": 1875, + "selected_both_wrong": 338 + }, + { + "budget": 0.3, + "threshold": -0.20665131509304047, + "operator": "ge", + "actual_fraction": 0.3125832223701731, + "metric": 0.24887307290721358, + "rescued": 920, + "harmed": 551, + "missed_rescue": 278, + "avoided_harm": 1558, + "selected_both_correct": 1942, + "selected_both_wrong": 343 + }, + { + "budget": 0.31, + "threshold": -0.2492215782403946, + "operator": "ge", + "actual_fraction": 0.3255659121171771, + "metric": 0.2585668315825877, + "rescued": 949, + "harmed": 576, + "missed_rescue": 249, + "avoided_harm": 1533, + "selected_both_correct": 2032, + "selected_both_wrong": 355 + }, + { + "budget": 0.32, + "threshold": -0.2695646286010742, + "operator": "ge", + "actual_fraction": 0.3312250332889481, + "metric": 0.2593193107424451, + "rescued": 960, + "harmed": 588, + "missed_rescue": 238, + "avoided_harm": 1521, + "selected_both_correct": 2073, + "selected_both_wrong": 359 + }, + { + "budget": 0.33, + "threshold": -0.2922053933143616, + "operator": "ge", + "actual_fraction": 0.340878828229028, + "metric": 0.2606628293353942, + "rescued": 978, + "harmed": 607, + "missed_rescue": 220, + "avoided_harm": 1502, + "selected_both_correct": 2145, + "selected_both_wrong": 366 + }, + { + "budget": 0.34, + "threshold": -0.32532864809036255, + "operator": "ge", + "actual_fraction": 0.3545272969374168, + "metric": 0.2655554337560316, + "rescued": 1009, + "harmed": 634, + "missed_rescue": 189, + "avoided_harm": 1475, + "selected_both_correct": 2239, + "selected_both_wrong": 378 + }, + { + "budget": 0.35, + "threshold": -0.33954745531082153, + "operator": "ge", + "actual_fraction": 0.35652463382157124, + "metric": 0.2654311640084225, + "rescued": 1013, + "harmed": 637, + "missed_rescue": 185, + "avoided_harm": 1472, + "selected_both_correct": 2254, + "selected_both_wrong": 380 + }, + { + "budget": 0.36, + "threshold": -0.3620870113372803, + "operator": "ge", + "actual_fraction": 0.36118508655126497, + "metric": 0.26164580206044635, + "rescued": 1022, + "harmed": 650, + "missed_rescue": 176, + "avoided_harm": 1459, + "selected_both_correct": 2286, + "selected_both_wrong": 382 + }, + { + "budget": 0.37, + "threshold": -0.4306718707084656, + "operator": "ge", + "actual_fraction": 0.37716378162450065, + "metric": 0.2617128175035257, + "rescued": 1048, + "harmed": 680, + "missed_rescue": 150, + "avoided_harm": 1429, + "selected_both_correct": 2406, + "selected_both_wrong": 398 + }, + { + "budget": 0.38, + "threshold": -0.4907958507537842, + "operator": "ge", + "actual_fraction": 0.38681757656458055, + "metric": 0.263551409505886, + "rescued": 1065, + "harmed": 698, + "missed_rescue": 133, + "avoided_harm": 1411, + "selected_both_correct": 2477, + "selected_both_wrong": 408 + }, + { + "budget": 0.39, + "threshold": -0.5423566699028015, + "operator": "ge", + "actual_fraction": 0.3914780292942743, + "metric": 0.25884391705053894, + "rescued": 1073, + "harmed": 706, + "missed_rescue": 125, + "avoided_harm": 1403, + "selected_both_correct": 2513, + "selected_both_wrong": 412 + }, + { + "budget": 0.4, + "threshold": -0.6501604318618774, + "operator": "ge", + "actual_fraction": 0.4014647137150466, + "metric": 0.2585252819028166, + "rescued": 1088, + "harmed": 722, + "missed_rescue": 110, + "avoided_harm": 1387, + "selected_both_correct": 2591, + "selected_both_wrong": 423 + }, + { + "budget": 0.41, + "threshold": -0.7341160178184509, + "operator": "ge", + "actual_fraction": 0.409121171770972, + "metric": 0.25763633403282094, + "rescued": 1099, + "harmed": 736, + "missed_rescue": 99, + "avoided_harm": 1373, + "selected_both_correct": 2648, + "selected_both_wrong": 433 + }, + { + "budget": 0.42, + "threshold": -0.8236439824104309, + "operator": "ge", + "actual_fraction": 0.414780292942743, + "metric": 0.2580877385441545, + "rescued": 1107, + "harmed": 749, + "missed_rescue": 91, + "avoided_harm": 1360, + "selected_both_correct": 2686, + "selected_both_wrong": 442 + }, + { + "budget": 0.43, + "threshold": -0.9715824723243713, + "operator": "ge", + "actual_fraction": 0.4247669773635153, + "metric": 0.2577978015300326, + "rescued": 1121, + "harmed": 767, + "missed_rescue": 77, + "avoided_harm": 1342, + "selected_both_correct": 2761, + "selected_both_wrong": 455 + }, + { + "budget": 0.44, + "threshold": -1.0671179294586182, + "operator": "ge", + "actual_fraction": 0.4330892143808256, + "metric": 0.2583375747408947, + "rescued": 1127, + "harmed": 782, + "missed_rescue": 71, + "avoided_harm": 1327, + "selected_both_correct": 2826, + "selected_both_wrong": 469 + }, + { + "budget": 0.45, + "threshold": -1.2702796459197998, + "operator": "ge", + "actual_fraction": 0.44241011984021306, + "metric": 0.2584292503515107, + "rescued": 1135, + "harmed": 794, + "missed_rescue": 63, + "avoided_harm": 1315, + "selected_both_correct": 2899, + "selected_both_wrong": 488 + }, + { + "budget": 0.46, + "threshold": -1.5780962705612183, + "operator": "ge", + "actual_fraction": 0.4517310252996005, + "metric": 0.2597048496818082, + "rescued": 1144, + "harmed": 806, + "missed_rescue": 54, + "avoided_harm": 1303, + "selected_both_correct": 2977, + "selected_both_wrong": 501 + }, + { + "budget": 0.47, + "threshold": -2.1831114292144775, + "operator": "ge", + "actual_fraction": 0.46504660452729696, + "metric": 0.2602262189251431, + "rescued": 1156, + "harmed": 833, + "missed_rescue": 42, + "avoided_harm": 1276, + "selected_both_correct": 3086, + "selected_both_wrong": 513 + }, + { + "budget": 0.48, + "threshold": -2.363466501235962, + "operator": "ge", + "actual_fraction": 0.46870838881491345, + "metric": 0.2585860131914839, + "rescued": 1156, + "harmed": 841, + "missed_rescue": 42, + "avoided_harm": 1268, + "selected_both_correct": 3121, + "selected_both_wrong": 514 + }, + { + "budget": 0.49, + "threshold": -2.523495674133301, + "operator": "ge", + "actual_fraction": 0.47203728362183756, + "metric": 0.2582183145054675, + "rescued": 1158, + "harmed": 850, + "missed_rescue": 40, + "avoided_harm": 1259, + "selected_both_correct": 3149, + "selected_both_wrong": 515 + }, + { + "budget": 0.5, + "threshold": -2.882524013519287, + "operator": "ge", + "actual_fraction": 0.4910119840213049, + "metric": 0.2537565056374797, + "rescued": 1167, + "harmed": 895, + "missed_rescue": 31, + "avoided_harm": 1214, + "selected_both_correct": 3319, + "selected_both_wrong": 519 + }, + { + "budget": 0.51, + "threshold": -3.2156200408935547, + "operator": "ge", + "actual_fraction": 0.5079893475366178, + "metric": 0.24446355185137025, + "rescued": 1171, + "harmed": 948, + "missed_rescue": 27, + "avoided_harm": 1161, + "selected_both_correct": 3464, + "selected_both_wrong": 521 + }, + { + "budget": 0.52, + "threshold": -4.331794738769531, + "operator": "ge", + "actual_fraction": 0.5149800266311585, + "metric": 0.24166288036554917, + "rescued": 1171, + "harmed": 970, + "missed_rescue": 27, + "avoided_harm": 1139, + "selected_both_correct": 3526, + "selected_both_wrong": 521 + }, + { + "budget": 0.53, + "threshold": -4.379722595214844, + "operator": "ge", + "actual_fraction": 0.5236351531291611, + "metric": 0.24052902234188292, + "rescued": 1171, + "harmed": 996, + "missed_rescue": 27, + "avoided_harm": 1113, + "selected_both_correct": 3604, + "selected_both_wrong": 521 + }, + { + "budget": 0.54, + "threshold": -4.431995391845703, + "operator": "ge", + "actual_fraction": 0.5352862849533955, + "metric": 0.23890974886692054, + "rescued": 1171, + "harmed": 1027, + "missed_rescue": 27, + "avoided_harm": 1082, + "selected_both_correct": 3713, + "selected_both_wrong": 521 + }, + { + "budget": 0.55, + "threshold": -4.467194557189941, + "operator": "ge", + "actual_fraction": 0.5449400798934754, + "metric": 0.23769524580634058, + "rescued": 1171, + "harmed": 1052, + "missed_rescue": 27, + "avoided_harm": 1057, + "selected_both_correct": 3803, + "selected_both_wrong": 522 + }, + { + "budget": 0.56, + "threshold": -4.493173122406006, + "operator": "ge", + "actual_fraction": 0.5565912117177098, + "metric": 0.23585027057526164, + "rescued": 1171, + "harmed": 1081, + "missed_rescue": 27, + "avoided_harm": 1028, + "selected_both_correct": 3913, + "selected_both_wrong": 523 + }, + { + "budget": 0.57, + "threshold": -4.505556106567383, + "operator": "ge", + "actual_fraction": 0.5629161118508655, + "metric": 0.23516386999345149, + "rescued": 1171, + "harmed": 1098, + "missed_rescue": 27, + "avoided_harm": 1011, + "selected_both_correct": 3972, + "selected_both_wrong": 523 + }, + { + "budget": 0.58, + "threshold": -4.531030654907227, + "operator": "ge", + "actual_fraction": 0.5752330226364847, + "metric": 0.2346874649726572, + "rescued": 1172, + "harmed": 1126, + "missed_rescue": 26, + "avoided_harm": 983, + "selected_both_correct": 4091, + "selected_both_wrong": 523 + }, + { + "budget": 0.59, + "threshold": -4.553508758544922, + "operator": "ge", + "actual_fraction": 0.5872170439414115, + "metric": 0.2344405533731278, + "rescued": 1174, + "harmed": 1153, + "missed_rescue": 24, + "avoided_harm": 956, + "selected_both_correct": 4205, + "selected_both_wrong": 524 + }, + { + "budget": 0.6, + "threshold": -4.568713188171387, + "operator": "ge", + "actual_fraction": 0.5968708388814914, + "metric": 0.23298977239960839, + "rescued": 1174, + "harmed": 1181, + "missed_rescue": 24, + "avoided_harm": 928, + "selected_both_correct": 4292, + "selected_both_wrong": 525 + }, + { + "budget": 0.61, + "threshold": -4.596630096435547, + "operator": "ge", + "actual_fraction": 0.6151797603195739, + "metric": 0.23298377792675512, + "rescued": 1176, + "harmed": 1222, + "missed_rescue": 22, + "avoided_harm": 887, + "selected_both_correct": 4469, + "selected_both_wrong": 525 + }, + { + "budget": 0.62, + "threshold": -4.6132402420043945, + "operator": "ge", + "actual_fraction": 0.625832223701731, + "metric": 0.23210994662774267, + "rescued": 1177, + "harmed": 1247, + "missed_rescue": 21, + "avoided_harm": 862, + "selected_both_correct": 4571, + "selected_both_wrong": 525 + }, + { + "budget": 0.63, + "threshold": -4.62782621383667, + "operator": "ge", + "actual_fraction": 0.6304926764314248, + "metric": 0.23218996917144671, + "rescued": 1178, + "harmed": 1257, + "missed_rescue": 20, + "avoided_harm": 852, + "selected_both_correct": 4616, + "selected_both_wrong": 525 + }, + { + "budget": 0.64, + "threshold": -4.642134666442871, + "operator": "ge", + "actual_fraction": 0.6391478029294274, + "metric": 0.23142911249669243, + "rescued": 1178, + "harmed": 1283, + "missed_rescue": 20, + "avoided_harm": 826, + "selected_both_correct": 4694, + "selected_both_wrong": 525 + }, + { + "budget": 0.65, + "threshold": -4.669188976287842, + "operator": "ge", + "actual_fraction": 0.6531291611185086, + "metric": 0.23641015321991868, + "rescued": 1180, + "harmed": 1308, + "missed_rescue": 18, + "avoided_harm": 801, + "selected_both_correct": 4834, + "selected_both_wrong": 526 + }, + { + "budget": 0.66, + "threshold": -4.687463760375977, + "operator": "ge", + "actual_fraction": 0.6614513981358189, + "metric": 0.2364194531367214, + "rescued": 1181, + "harmed": 1330, + "missed_rescue": 17, + "avoided_harm": 779, + "selected_both_correct": 4910, + "selected_both_wrong": 527 + }, + { + "budget": 0.67, + "threshold": -4.702577590942383, + "operator": "ge", + "actual_fraction": 0.668774966711052, + "metric": 0.2343963356223511, + "rescued": 1181, + "harmed": 1345, + "missed_rescue": 17, + "avoided_harm": 764, + "selected_both_correct": 4983, + "selected_both_wrong": 527 + }, + { + "budget": 0.68, + "threshold": -4.743311405181885, + "operator": "ge", + "actual_fraction": 0.6844207723035952, + "metric": 0.23042848856529904, + "rescued": 1181, + "harmed": 1386, + "missed_rescue": 17, + "avoided_harm": 723, + "selected_both_correct": 5130, + "selected_both_wrong": 527 + }, + { + "budget": 0.69, + "threshold": -4.790846824645996, + "operator": "ge", + "actual_fraction": 0.6977363515312917, + "metric": 0.22668682410501836, + "rescued": 1181, + "harmed": 1421, + "missed_rescue": 17, + "avoided_harm": 688, + "selected_both_correct": 5255, + "selected_both_wrong": 527 + }, + { + "budget": 0.7, + "threshold": -4.810942649841309, + "operator": "ge", + "actual_fraction": 0.7030625832223701, + "metric": 0.224858564156682, + "rescued": 1181, + "harmed": 1435, + "missed_rescue": 17, + "avoided_harm": 674, + "selected_both_correct": 5305, + "selected_both_wrong": 527 + }, + { + "budget": 0.71, + "threshold": -4.89448356628418, + "operator": "ge", + "actual_fraction": 0.7177097203728362, + "metric": 0.22495430153906848, + "rescued": 1184, + "harmed": 1474, + "missed_rescue": 14, + "avoided_harm": 635, + "selected_both_correct": 5439, + "selected_both_wrong": 527 + }, + { + "budget": 0.72, + "threshold": -4.957235336303711, + "operator": "ge", + "actual_fraction": 0.7330226364846871, + "metric": 0.2227723141589208, + "rescued": 1185, + "harmed": 1508, + "missed_rescue": 13, + "avoided_harm": 601, + "selected_both_correct": 5587, + "selected_both_wrong": 528 + }, + { + "budget": 0.73, + "threshold": -4.993195533752441, + "operator": "ge", + "actual_fraction": 0.7400133155792277, + "metric": 0.22088760969867527, + "rescued": 1185, + "harmed": 1526, + "missed_rescue": 13, + "avoided_harm": 583, + "selected_both_correct": 5653, + "selected_both_wrong": 528 + }, + { + "budget": 0.74, + "threshold": -5.04953145980835, + "operator": "ge", + "actual_fraction": 0.7536617842876165, + "metric": 0.22079705724095997, + "rescued": 1187, + "harmed": 1562, + "missed_rescue": 11, + "avoided_harm": 547, + "selected_both_correct": 5779, + "selected_both_wrong": 528 + }, + { + "budget": 0.75, + "threshold": -5.097166538238525, + "operator": "ge", + "actual_fraction": 0.7673102529960053, + "metric": 0.22021046042527834, + "rescued": 1188, + "harmed": 1606, + "missed_rescue": 10, + "avoided_harm": 503, + "selected_both_correct": 5898, + "selected_both_wrong": 528 + }, + { + "budget": 0.76, + "threshold": -5.135637283325195, + "operator": "ge", + "actual_fraction": 0.7752996005326231, + "metric": 0.21890697421942173, + "rescued": 1189, + "harmed": 1630, + "missed_rescue": 9, + "avoided_harm": 479, + "selected_both_correct": 5968, + "selected_both_wrong": 529 + }, + { + "budget": 0.77, + "threshold": -5.158949375152588, + "operator": "ge", + "actual_fraction": 0.7802929427430093, + "metric": 0.21872915571848423, + "rescued": 1190, + "harmed": 1644, + "missed_rescue": 8, + "avoided_harm": 465, + "selected_both_correct": 6013, + "selected_both_wrong": 529 + }, + { + "budget": 0.78, + "threshold": -5.1902618408203125, + "operator": "ge", + "actual_fraction": 0.7869507323568575, + "metric": 0.22218423271635374, + "rescued": 1192, + "harmed": 1664, + "missed_rescue": 6, + "avoided_harm": 445, + "selected_both_correct": 6071, + "selected_both_wrong": 529 + }, + { + "budget": 0.79, + "threshold": -5.255657196044922, + "operator": "ge", + "actual_fraction": 0.7976031957390146, + "metric": 0.22528188494939438, + "rescued": 1193, + "harmed": 1694, + "missed_rescue": 5, + "avoided_harm": 415, + "selected_both_correct": 6168, + "selected_both_wrong": 529 + }, + { + "budget": 0.8, + "threshold": -5.313392162322998, + "operator": "ge", + "actual_fraction": 0.8095872170439414, + "metric": 0.2243556727732779, + "rescued": 1193, + "harmed": 1723, + "missed_rescue": 5, + "avoided_harm": 386, + "selected_both_correct": 6281, + "selected_both_wrong": 531 + }, + { + "budget": 0.81, + "threshold": -5.414909362792969, + "operator": "ge", + "actual_fraction": 0.8185752330226365, + "metric": 0.22787110610871256, + "rescued": 1194, + "harmed": 1743, + "missed_rescue": 4, + "avoided_harm": 366, + "selected_both_correct": 6368, + "selected_both_wrong": 531 + }, + { + "budget": 0.82, + "threshold": -12.358579635620117, + "operator": "ge", + "actual_fraction": 0.8305592543275633, + "metric": 0.22766731851915514, + "rescued": 1195, + "harmed": 1768, + "missed_rescue": 3, + "avoided_harm": 341, + "selected_both_correct": 6486, + "selected_both_wrong": 531 + }, + { + "budget": 0.83, + "threshold": -12.387406349182129, + "operator": "ge", + "actual_fraction": 0.8418774966711052, + "metric": 0.22724332849465062, + "rescued": 1195, + "harmed": 1791, + "missed_rescue": 3, + "avoided_harm": 318, + "selected_both_correct": 6599, + "selected_both_wrong": 531 + }, + { + "budget": 0.84, + "threshold": -12.407716751098633, + "operator": "ge", + "actual_fraction": 0.855525965379494, + "metric": 0.226744842709326, + "rescued": 1195, + "harmed": 1822, + "missed_rescue": 3, + "avoided_harm": 287, + "selected_both_correct": 6732, + "selected_both_wrong": 531 + }, + { + "budget": 0.85, + "threshold": -12.423635482788086, + "operator": "ge", + "actual_fraction": 0.8655126498002663, + "metric": 0.22649489375331527, + "rescued": 1195, + "harmed": 1836, + "missed_rescue": 3, + "avoided_harm": 273, + "selected_both_correct": 6838, + "selected_both_wrong": 531 + }, + { + "budget": 0.86, + "threshold": -12.440216064453125, + "operator": "ge", + "actual_fraction": 0.8745006657789614, + "metric": 0.2257220122343147, + "rescued": 1195, + "harmed": 1853, + "missed_rescue": 3, + "avoided_harm": 256, + "selected_both_correct": 6929, + "selected_both_wrong": 531 + }, + { + "budget": 0.87, + "threshold": -12.449625015258789, + "operator": "ge", + "actual_fraction": 0.8788282290279628, + "metric": 0.22557232485094011, + "rescued": 1195, + "harmed": 1862, + "missed_rescue": 3, + "avoided_harm": 247, + "selected_both_correct": 6972, + "selected_both_wrong": 531 + }, + { + "budget": 0.88, + "threshold": -12.47104263305664, + "operator": "ge", + "actual_fraction": 0.8894806924101198, + "metric": 0.22526395729914153, + "rescued": 1195, + "harmed": 1880, + "missed_rescue": 3, + "avoided_harm": 229, + "selected_both_correct": 7082, + "selected_both_wrong": 531 + }, + { + "budget": 0.89, + "threshold": -12.487972259521484, + "operator": "ge", + "actual_fraction": 0.8984687083888149, + "metric": 0.22454513932959558, + "rescued": 1195, + "harmed": 1895, + "missed_rescue": 3, + "avoided_harm": 214, + "selected_both_correct": 7174, + "selected_both_wrong": 532 + }, + { + "budget": 0.9, + "threshold": -12.537336349487305, + "operator": "ge", + "actual_fraction": 0.9174434087882823, + "metric": 0.22750863517366876, + "rescued": 1196, + "harmed": 1936, + "missed_rescue": 2, + "avoided_harm": 173, + "selected_both_correct": 7360, + "selected_both_wrong": 532 + }, + { + "budget": 0.91, + "threshold": -12.567386627197266, + "operator": "ge", + "actual_fraction": 0.9257656458055925, + "metric": 0.2269338977156315, + "rescued": 1196, + "harmed": 1952, + "missed_rescue": 2, + "avoided_harm": 157, + "selected_both_correct": 7444, + "selected_both_wrong": 532 + }, + { + "budget": 0.92, + "threshold": -12.593729972839355, + "operator": "ge", + "actual_fraction": 0.9354194407456724, + "metric": 0.2261662288100518, + "rescued": 1196, + "harmed": 1974, + "missed_rescue": 2, + "avoided_harm": 135, + "selected_both_correct": 7538, + "selected_both_wrong": 532 + }, + { + "budget": 0.93, + "threshold": -12.62214469909668, + "operator": "ge", + "actual_fraction": 0.9447403462050599, + "metric": 0.22575388032654248, + "rescued": 1196, + "harmed": 1990, + "missed_rescue": 2, + "avoided_harm": 119, + "selected_both_correct": 7634, + "selected_both_wrong": 532 + }, + { + "budget": 0.94, + "threshold": -12.674238204956055, + "operator": "ge", + "actual_fraction": 0.9563914780292943, + "metric": 0.2247267186715921, + "rescued": 1196, + "harmed": 2017, + "missed_rescue": 2, + "avoided_harm": 92, + "selected_both_correct": 7747, + "selected_both_wrong": 532 + }, + { + "budget": 0.95, + "threshold": -12.761373519897461, + "operator": "ge", + "actual_fraction": 0.9660452729693741, + "metric": 0.2231728323690651, + "rescued": 1196, + "harmed": 2035, + "missed_rescue": 2, + "avoided_harm": 74, + "selected_both_correct": 7845, + "selected_both_wrong": 532 + }, + { + "budget": 0.96, + "threshold": -12.814491271972656, + "operator": "ge", + "actual_fraction": 0.9707057256990679, + "metric": 0.22274733924762188, + "rescued": 1196, + "harmed": 2042, + "missed_rescue": 2, + "avoided_harm": 67, + "selected_both_correct": 7894, + "selected_both_wrong": 532 + }, + { + "budget": 0.97, + "threshold": -12.931108474731445, + "operator": "ge", + "actual_fraction": 0.9773635153129161, + "metric": 0.22197615401784254, + "rescued": 1196, + "harmed": 2056, + "missed_rescue": 2, + "avoided_harm": 53, + "selected_both_correct": 7960, + "selected_both_wrong": 532 + }, + { + "budget": 0.98, + "threshold": -12.97988510131836, + "operator": "ge", + "actual_fraction": 0.9833555259653795, + "metric": 0.22185235756944024, + "rescued": 1196, + "harmed": 2065, + "missed_rescue": 2, + "avoided_harm": 44, + "selected_both_correct": 8023, + "selected_both_wrong": 532 + }, + { + "budget": 0.99, + "threshold": -13.074522018432617, + "operator": "ge", + "actual_fraction": 0.9963382157123835, + "metric": 0.2206072711925277, + "rescued": 1197, + "harmed": 2103, + "missed_rescue": 1, + "avoided_harm": 6, + "selected_both_correct": 8139, + "selected_both_wrong": 533 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.22361115727328065, + "rescued": 1198, + "harmed": 2109, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 8176, + "selected_both_wrong": 533 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 1891, + "rescuable": 286, + "harmful": 518, + "both_wrong": 109, + "unit": "event", + "total": 2804 + }, + "test": { + "both_correct": 8176, + "rescuable": 1198, + "harmful": 2109, + "both_wrong": 533, + "unit": "event", + "total": 12016 + } + }, + "small": { + "validation": 0.18254072168965788, + "test": 0.18129624718318377 + }, + "large": { + "validation": 0.20666016863629594, + "test": 0.22361115727328065 + }, + "learned_minus_confidence_test": 0.0059136067749739896, + "learned_minus_small_test": 0.07692206732228374, + "learned_minus_large_test": 0.03460715723218685 + }, + { + "key": [ + "construction", + "yolo26x", + "Qwen3.8" + ], + "expert_display": "YOLO26x", + "run": "shared_12_yolo26x_Qwen3.8", + "validation_rows": 701, + "test_rows": 3004, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detection_count", + "has_detections", + "max_score", + "mean_score", + "detection_count_valid", + "has_detections_valid", + "max_score_valid", + "mean_score_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/yolo26x/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 93, + "validation_area_0_25": 0.17942553980644182, + "validation_point": { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2642878791936722, + "rescued": 288, + "harmed": 202, + "missed_rescue": 19, + "avoided_harm": 18, + "selected_both_correct": 2035, + "selected_both_wrong": 79 + }, + "test_point": { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9221038615179761, + "metric": 0.2597490004762105, + "rescued": 1180, + "harmed": 728, + "missed_rescue": 102, + "avoided_harm": 49, + "selected_both_correct": 8774, + "selected_both_wrong": 398 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": 1.0, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.1728507294858811, + "rescued": 0, + "harmed": 34, + "missed_rescue": 307, + "avoided_harm": 186, + "selected_both_correct": 498, + "selected_both_wrong": 0 + }, + { + "budget": 0.2, + "threshold": -0.0010652542114257812, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.1728507294858811, + "rescued": 0, + "harmed": 34, + "missed_rescue": 307, + "avoided_harm": 186, + "selected_both_correct": 526, + "selected_both_wrong": 0 + }, + { + "budget": 0.21, + "threshold": -0.0011425018310546875, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.17237009889855048, + "rescued": 0, + "harmed": 36, + "missed_rescue": 307, + "avoided_harm": 184, + "selected_both_correct": 552, + "selected_both_wrong": 0 + }, + { + "budget": 0.22, + "threshold": -0.0011968612670898438, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.17209204025345443, + "rescued": 0, + "harmed": 37, + "missed_rescue": 307, + "avoided_harm": 183, + "selected_both_correct": 578, + "selected_both_wrong": 1 + }, + { + "budget": 0.23, + "threshold": -0.0012493133544921875, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.1716297194027841, + "rescued": 0, + "harmed": 39, + "missed_rescue": 307, + "avoided_harm": 181, + "selected_both_correct": 604, + "selected_both_wrong": 1 + }, + { + "budget": 0.24, + "threshold": -0.0013513565063476562, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.17076888726505096, + "rescued": 0, + "harmed": 42, + "missed_rescue": 307, + "avoided_harm": 178, + "selected_both_correct": 629, + "selected_both_wrong": 1 + }, + { + "budget": 0.25, + "threshold": -0.0015010833740234375, + "operator": "gt", + "actual_fraction": 0.24821683309557774, + "metric": 0.17076888726505096, + "rescued": 0, + "harmed": 42, + "missed_rescue": 307, + "avoided_harm": 178, + "selected_both_correct": 652, + "selected_both_wrong": 2 + }, + { + "budget": 0.26, + "threshold": -0.00157928466796875, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.17010751160367527, + "rescued": 0, + "harmed": 45, + "missed_rescue": 307, + "avoided_harm": 175, + "selected_both_correct": 681, + "selected_both_wrong": 2 + }, + { + "budget": 0.27, + "threshold": -0.0017137527465820312, + "operator": "gt", + "actual_fraction": 0.268188302425107, + "metric": 0.17010751160367527, + "rescued": 0, + "harmed": 45, + "missed_rescue": 307, + "avoided_harm": 175, + "selected_both_correct": 705, + "selected_both_wrong": 2 + }, + { + "budget": 0.28, + "threshold": -0.0017614364624023438, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.16948188597804964, + "rescued": 0, + "harmed": 48, + "missed_rescue": 307, + "avoided_harm": 172, + "selected_both_correct": 734, + "selected_both_wrong": 2 + }, + { + "budget": 0.29, + "threshold": -0.0018463134765625, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.16948188597804964, + "rescued": 0, + "harmed": 48, + "missed_rescue": 307, + "avoided_harm": 172, + "selected_both_correct": 762, + "selected_both_wrong": 2 + }, + { + "budget": 0.3, + "threshold": -0.001934051513671875, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.16880965944961934, + "rescued": 0, + "harmed": 51, + "missed_rescue": 307, + "avoided_harm": 169, + "selected_both_correct": 787, + "selected_both_wrong": 2 + }, + { + "budget": 0.31, + "threshold": -0.0020427703857421875, + "operator": "gt", + "actual_fraction": 0.30813124108416545, + "metric": 0.16830460894456886, + "rescued": 0, + "harmed": 56, + "missed_rescue": 307, + "avoided_harm": 164, + "selected_both_correct": 806, + "selected_both_wrong": 2 + }, + { + "budget": 0.32, + "threshold": -0.0021495819091796875, + "operator": "gt", + "actual_fraction": 0.3181169757489301, + "metric": 0.16811053970066264, + "rescued": 0, + "harmed": 57, + "missed_rescue": 307, + "avoided_harm": 163, + "selected_both_correct": 833, + "selected_both_wrong": 2 + }, + { + "budget": 0.33, + "threshold": -0.002208709716796875, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.1671811299174608, + "rescued": 0, + "harmed": 61, + "missed_rescue": 307, + "avoided_harm": 159, + "selected_both_correct": 861, + "selected_both_wrong": 2 + }, + { + "budget": 0.34, + "threshold": -0.0023326873779296875, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.16645634059427164, + "rescued": 0, + "harmed": 64, + "missed_rescue": 307, + "avoided_harm": 156, + "selected_both_correct": 886, + "selected_both_wrong": 2 + }, + { + "budget": 0.35, + "threshold": -0.0024623870849609375, + "operator": "gt", + "actual_fraction": 0.34807417974322397, + "metric": 0.16600354762332126, + "rescued": 0, + "harmed": 66, + "missed_rescue": 307, + "avoided_harm": 154, + "selected_both_correct": 908, + "selected_both_wrong": 2 + }, + { + "budget": 0.36, + "threshold": -0.0025615692138671875, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.16502549737843858, + "rescued": 0, + "harmed": 70, + "missed_rescue": 307, + "avoided_harm": 150, + "selected_both_correct": 936, + "selected_both_wrong": 2 + }, + { + "budget": 0.37, + "threshold": -0.0026721954345703125, + "operator": "gt", + "actual_fraction": 0.3666191155492154, + "metric": 0.16467825104188744, + "rescued": 0, + "harmed": 72, + "missed_rescue": 307, + "avoided_harm": 148, + "selected_both_correct": 954, + "selected_both_wrong": 2 + }, + { + "budget": 0.38, + "threshold": -0.0028228759765625, + "operator": "gt", + "actual_fraction": 0.37660485021398005, + "metric": 0.16467825104188744, + "rescued": 0, + "harmed": 76, + "missed_rescue": 307, + "avoided_harm": 144, + "selected_both_correct": 978, + "selected_both_wrong": 2 + }, + { + "budget": 0.39, + "threshold": -0.0029354095458984375, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.16749463738508683, + "rescued": 1, + "harmed": 81, + "missed_rescue": 306, + "avoided_harm": 139, + "selected_both_correct": 1008, + "selected_both_wrong": 2 + }, + { + "budget": 0.4, + "threshold": -0.00307464599609375, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.16607474321791724, + "rescued": 1, + "harmed": 87, + "missed_rescue": 306, + "avoided_harm": 133, + "selected_both_correct": 1030, + "selected_both_wrong": 2 + }, + { + "budget": 0.41, + "threshold": -0.0033626556396484375, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.16514206572141965, + "rescued": 1, + "harmed": 90, + "missed_rescue": 306, + "avoided_harm": 130, + "selected_both_correct": 1055, + "selected_both_wrong": 2 + }, + { + "budget": 0.42, + "threshold": -0.0035381317138671875, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.16444252141114554, + "rescued": 1, + "harmed": 93, + "missed_rescue": 306, + "avoided_harm": 127, + "selected_both_correct": 1080, + "selected_both_wrong": 2 + }, + { + "budget": 0.43, + "threshold": -0.003765106201171875, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.1639139801849299, + "rescued": 1, + "harmed": 96, + "missed_rescue": 306, + "avoided_harm": 124, + "selected_both_correct": 1105, + "selected_both_wrong": 2 + }, + { + "budget": 0.44, + "threshold": -0.00392913818359375, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.16237039262040495, + "rescued": 1, + "harmed": 103, + "missed_rescue": 306, + "avoided_harm": 117, + "selected_both_correct": 1126, + "selected_both_wrong": 2 + }, + { + "budget": 0.45, + "threshold": -0.004085540771484375, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.16187473569847557, + "rescued": 1, + "harmed": 105, + "missed_rescue": 306, + "avoided_harm": 115, + "selected_both_correct": 1152, + "selected_both_wrong": 2 + }, + { + "budget": 0.46, + "threshold": -0.004364013671875, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.1640341119762699, + "rescued": 2, + "harmed": 110, + "missed_rescue": 305, + "avoided_harm": 110, + "selected_both_correct": 1174, + "selected_both_wrong": 2 + }, + { + "budget": 0.47, + "threshold": -0.004680633544921875, + "operator": "gt", + "actual_fraction": 0.46790299572039945, + "metric": 0.16656779979633995, + "rescued": 3, + "harmed": 113, + "missed_rescue": 304, + "avoided_harm": 107, + "selected_both_correct": 1194, + "selected_both_wrong": 2 + }, + { + "budget": 0.48, + "threshold": -0.00490570068359375, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.16532395129425315, + "rescued": 3, + "harmed": 119, + "missed_rescue": 304, + "avoided_harm": 101, + "selected_both_correct": 1220, + "selected_both_wrong": 2 + }, + { + "budget": 0.49, + "threshold": -0.005039215087890625, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.16467625831496033, + "rescued": 5, + "harmed": 123, + "missed_rescue": 302, + "avoided_harm": 97, + "selected_both_correct": 1241, + "selected_both_wrong": 3 + }, + { + "budget": 0.5, + "threshold": -0.005321502685546875, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.16569589650413527, + "rescued": 11, + "harmed": 128, + "missed_rescue": 296, + "avoided_harm": 92, + "selected_both_correct": 1257, + "selected_both_wrong": 4 + }, + { + "budget": 0.51, + "threshold": -0.005619049072265625, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.1671927745457157, + "rescued": 18, + "harmed": 129, + "missed_rescue": 289, + "avoided_harm": 91, + "selected_both_correct": 1276, + "selected_both_wrong": 5 + }, + { + "budget": 0.52, + "threshold": -0.00586700439453125, + "operator": "gt", + "actual_fraction": 0.5178316690442225, + "metric": 0.16789087940773334, + "rescued": 23, + "harmed": 131, + "missed_rescue": 284, + "avoided_harm": 89, + "selected_both_correct": 1293, + "selected_both_wrong": 5 + }, + { + "budget": 0.53, + "threshold": -0.00605010986328125, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.16946103210148153, + "rescued": 30, + "harmed": 132, + "missed_rescue": 277, + "avoided_harm": 88, + "selected_both_correct": 1316, + "selected_both_wrong": 6 + }, + { + "budget": 0.54, + "threshold": -0.006439208984375, + "operator": "gt", + "actual_fraction": 0.5378031383737518, + "metric": 0.17052688690619724, + "rescued": 35, + "harmed": 133, + "missed_rescue": 272, + "avoided_harm": 87, + "selected_both_correct": 1334, + "selected_both_wrong": 6 + }, + { + "budget": 0.55, + "threshold": -0.006641387939453125, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.17103667850460152, + "rescued": 42, + "harmed": 134, + "missed_rescue": 265, + "avoided_harm": 86, + "selected_both_correct": 1358, + "selected_both_wrong": 6 + }, + { + "budget": 0.56, + "threshold": -0.0070648193359375, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.17291017019862398, + "rescued": 49, + "harmed": 135, + "missed_rescue": 258, + "avoided_harm": 85, + "selected_both_correct": 1376, + "selected_both_wrong": 8 + }, + { + "budget": 0.57, + "threshold": -0.007518768310546875, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.17468909977784217, + "rescued": 55, + "harmed": 135, + "missed_rescue": 252, + "avoided_harm": 85, + "selected_both_correct": 1397, + "selected_both_wrong": 9 + }, + { + "budget": 0.58, + "threshold": -0.00818634033203125, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.1754433851238893, + "rescued": 60, + "harmed": 136, + "missed_rescue": 247, + "avoided_harm": 84, + "selected_both_correct": 1416, + "selected_both_wrong": 12 + }, + { + "budget": 0.59, + "threshold": -0.00878143310546875, + "operator": "gt", + "actual_fraction": 0.5877318116975749, + "metric": 0.17567949038614306, + "rescued": 65, + "harmed": 140, + "missed_rescue": 242, + "avoided_harm": 80, + "selected_both_correct": 1430, + "selected_both_wrong": 13 + }, + { + "budget": 0.6, + "threshold": -0.00955963134765625, + "operator": "gt", + "actual_fraction": 0.5977175463623395, + "metric": 0.17726242688944494, + "rescued": 73, + "harmed": 142, + "missed_rescue": 234, + "avoided_harm": 78, + "selected_both_correct": 1448, + "selected_both_wrong": 13 + }, + { + "budget": 0.61, + "threshold": -0.0099334716796875, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.17927753989081613, + "rescued": 81, + "harmed": 143, + "missed_rescue": 226, + "avoided_harm": 77, + "selected_both_correct": 1471, + "selected_both_wrong": 13 + }, + { + "budget": 0.62, + "threshold": -0.010650634765625, + "operator": "gt", + "actual_fraction": 0.6176890156918687, + "metric": 0.1803426895170312, + "rescued": 86, + "harmed": 144, + "missed_rescue": 221, + "avoided_harm": 76, + "selected_both_correct": 1489, + "selected_both_wrong": 13 + }, + { + "budget": 0.63, + "threshold": -0.01129150390625, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.18256297675338154, + "rescued": 94, + "harmed": 145, + "missed_rescue": 213, + "avoided_harm": 75, + "selected_both_correct": 1509, + "selected_both_wrong": 16 + }, + { + "budget": 0.64, + "threshold": -0.011871337890625, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.18741054744283442, + "rescued": 103, + "harmed": 149, + "missed_rescue": 204, + "avoided_harm": 71, + "selected_both_correct": 1524, + "selected_both_wrong": 16 + }, + { + "budget": 0.65, + "threshold": -0.0125274658203125, + "operator": "gt", + "actual_fraction": 0.6476462196861626, + "metric": 0.18874453577558464, + "rescued": 109, + "harmed": 150, + "missed_rescue": 198, + "avoided_harm": 70, + "selected_both_correct": 1540, + "selected_both_wrong": 17 + }, + { + "budget": 0.66, + "threshold": -0.012969970703125, + "operator": "gt", + "actual_fraction": 0.6562054208273894, + "metric": 0.18973030032445837, + "rescued": 114, + "harmed": 152, + "missed_rescue": 193, + "avoided_harm": 68, + "selected_both_correct": 1556, + "selected_both_wrong": 18 + }, + { + "budget": 0.67, + "threshold": -0.01363372802734375, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.1910322331722189, + "rescued": 122, + "harmed": 157, + "missed_rescue": 185, + "avoided_harm": 63, + "selected_both_correct": 1576, + "selected_both_wrong": 21 + }, + { + "budget": 0.68, + "threshold": -0.01473236083984375, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.19273815380416676, + "rescued": 128, + "harmed": 158, + "missed_rescue": 179, + "avoided_harm": 62, + "selected_both_correct": 1595, + "selected_both_wrong": 23 + }, + { + "budget": 0.69, + "threshold": -0.015716552734375, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.19362404817095658, + "rescued": 133, + "harmed": 161, + "missed_rescue": 174, + "avoided_harm": 59, + "selected_both_correct": 1612, + "selected_both_wrong": 26 + }, + { + "budget": 0.7, + "threshold": -0.017578125, + "operator": "gt", + "actual_fraction": 0.6975748930099858, + "metric": 0.19489832797986031, + "rescued": 138, + "harmed": 163, + "missed_rescue": 169, + "avoided_harm": 57, + "selected_both_correct": 1626, + "selected_both_wrong": 29 + }, + { + "budget": 0.71, + "threshold": -0.0189056396484375, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.19709243659719292, + "rescued": 147, + "harmed": 166, + "missed_rescue": 160, + "avoided_harm": 54, + "selected_both_correct": 1646, + "selected_both_wrong": 29 + }, + { + "budget": 0.72, + "threshold": -0.019866943359375, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.20000323882160753, + "rescued": 155, + "harmed": 166, + "missed_rescue": 152, + "avoided_harm": 54, + "selected_both_correct": 1666, + "selected_both_wrong": 29 + }, + { + "budget": 0.73, + "threshold": -0.020965576171875, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.2015690267335004, + "rescued": 160, + "harmed": 167, + "missed_rescue": 147, + "avoided_harm": 53, + "selected_both_correct": 1684, + "selected_both_wrong": 33 + }, + { + "budget": 0.74, + "threshold": -0.021453857421875, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.2034984942593638, + "rescued": 166, + "harmed": 168, + "missed_rescue": 141, + "avoided_harm": 52, + "selected_both_correct": 1703, + "selected_both_wrong": 35 + }, + { + "budget": 0.75, + "threshold": -0.0248870849609375, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.20679879821670866, + "rescued": 176, + "harmed": 170, + "missed_rescue": 131, + "avoided_harm": 50, + "selected_both_correct": 1718, + "selected_both_wrong": 36 + }, + { + "budget": 0.76, + "threshold": -0.0269622802734375, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.20972162293572166, + "rescued": 184, + "harmed": 171, + "missed_rescue": 123, + "avoided_harm": 49, + "selected_both_correct": 1735, + "selected_both_wrong": 38 + }, + { + "budget": 0.77, + "threshold": -0.0282135009765625, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.2126226551226551, + "rescued": 191, + "harmed": 174, + "missed_rescue": 116, + "avoided_harm": 46, + "selected_both_correct": 1752, + "selected_both_wrong": 39 + }, + { + "budget": 0.78, + "threshold": -0.0299835205078125, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.21516614155218836, + "rescued": 198, + "harmed": 175, + "missed_rescue": 109, + "avoided_harm": 45, + "selected_both_correct": 1771, + "selected_both_wrong": 40 + }, + { + "budget": 0.79, + "threshold": -0.033966064453125, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.2162880860385455, + "rescued": 204, + "harmed": 177, + "missed_rescue": 103, + "avoided_harm": 43, + "selected_both_correct": 1788, + "selected_both_wrong": 43 + }, + { + "budget": 0.8, + "threshold": -0.037200927734375, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.21779120256650406, + "rescued": 209, + "harmed": 179, + "missed_rescue": 98, + "avoided_harm": 41, + "selected_both_correct": 1806, + "selected_both_wrong": 46 + }, + { + "budget": 0.81, + "threshold": -0.04132080078125, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.2201843117420812, + "rescued": 216, + "harmed": 182, + "missed_rescue": 91, + "avoided_harm": 38, + "selected_both_correct": 1820, + "selected_both_wrong": 50 + }, + { + "budget": 0.82, + "threshold": -0.0455322265625, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.22188537527520574, + "rescued": 222, + "harmed": 184, + "missed_rescue": 85, + "avoided_harm": 36, + "selected_both_correct": 1838, + "selected_both_wrong": 52 + }, + { + "budget": 0.83, + "threshold": -0.051849365234375, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.2260242392475367, + "rescued": 231, + "harmed": 187, + "missed_rescue": 76, + "avoided_harm": 33, + "selected_both_correct": 1854, + "selected_both_wrong": 52 + }, + { + "budget": 0.84, + "threshold": -0.06878662109375, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.2282665192342907, + "rescued": 238, + "harmed": 190, + "missed_rescue": 69, + "avoided_harm": 30, + "selected_both_correct": 1869, + "selected_both_wrong": 55 + }, + { + "budget": 0.85, + "threshold": -0.08477783203125, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.23641538775702553, + "rescued": 246, + "harmed": 193, + "missed_rescue": 61, + "avoided_harm": 27, + "selected_both_correct": 1884, + "selected_both_wrong": 57 + }, + { + "budget": 0.86, + "threshold": -0.09912109375, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.2387681003514125, + "rescued": 251, + "harmed": 193, + "missed_rescue": 56, + "avoided_harm": 27, + "selected_both_correct": 1904, + "selected_both_wrong": 60 + }, + { + "budget": 0.87, + "threshold": -0.1298828125, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.24246700995212897, + "rescued": 258, + "harmed": 193, + "missed_rescue": 49, + "avoided_harm": 27, + "selected_both_correct": 1923, + "selected_both_wrong": 62 + }, + { + "budget": 0.88, + "threshold": -0.1441650390625, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.2423696527366133, + "rescued": 259, + "harmed": 197, + "missed_rescue": 48, + "avoided_harm": 23, + "selected_both_correct": 1941, + "selected_both_wrong": 67 + }, + { + "budget": 0.89, + "threshold": -0.182373046875, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.24478740599615373, + "rescued": 266, + "harmed": 199, + "missed_rescue": 41, + "avoided_harm": 21, + "selected_both_correct": 1959, + "selected_both_wrong": 68 + }, + { + "budget": 0.9, + "threshold": -0.2049560546875, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.25736315359477124, + "rescued": 273, + "harmed": 199, + "missed_rescue": 34, + "avoided_harm": 21, + "selected_both_correct": 1977, + "selected_both_wrong": 71 + }, + { + "budget": 0.91, + "threshold": -0.25830078125, + "operator": "gt", + "actual_fraction": 0.9072753209700428, + "metric": 0.2597551406926407, + "rescued": 278, + "harmed": 200, + "missed_rescue": 29, + "avoided_harm": 20, + "selected_both_correct": 1992, + "selected_both_wrong": 74 + }, + { + "budget": 0.92, + "threshold": -0.32080078125, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.2629241291812946, + "rescued": 284, + "harmed": 201, + "missed_rescue": 23, + "avoided_harm": 19, + "selected_both_correct": 2013, + "selected_both_wrong": 78 + }, + { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2642878791936722, + "rescued": 288, + "harmed": 202, + "missed_rescue": 19, + "avoided_harm": 18, + "selected_both_correct": 2035, + "selected_both_wrong": 79 + }, + { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.2639590174546567, + "rescued": 290, + "harmed": 203, + "missed_rescue": 17, + "avoided_harm": 17, + "selected_both_correct": 2057, + "selected_both_wrong": 82 + }, + { + "budget": 0.95, + "threshold": -0.611328125, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.26132796745180287, + "rescued": 293, + "harmed": 207, + "missed_rescue": 14, + "avoided_harm": 13, + "selected_both_correct": 2078, + "selected_both_wrong": 82 + }, + { + "budget": 0.96, + "threshold": -0.70068359375, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.2584345323365104, + "rescued": 296, + "harmed": 212, + "missed_rescue": 11, + "avoided_harm": 8, + "selected_both_correct": 2096, + "selected_both_wrong": 84 + }, + { + "budget": 0.97, + "threshold": -0.75048828125, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.2503146324574896, + "rescued": 299, + "harmed": 215, + "missed_rescue": 8, + "avoided_harm": 5, + "selected_both_correct": 2116, + "selected_both_wrong": 86 + }, + { + "budget": 0.98, + "threshold": -0.7841796875, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.2545437366865938, + "rescued": 302, + "harmed": 216, + "missed_rescue": 5, + "avoided_harm": 4, + "selected_both_correct": 2139, + "selected_both_wrong": 87 + }, + { + "budget": 0.99, + "threshold": -0.85107421875, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.2541444987255433, + "rescued": 304, + "harmed": 218, + "missed_rescue": 3, + "avoided_harm": 2, + "selected_both_correct": 2163, + "selected_both_wrong": 87 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.25448626222520154, + "rescued": 307, + "harmed": 220, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 2189, + "selected_both_wrong": 88 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": 1.0, + "operator": "ge", + "actual_fraction": 0.17942743009320905, + "metric": 0.1784481207469488, + "rescued": 4, + "harmed": 94, + "missed_rescue": 1278, + "avoided_harm": 683, + "selected_both_correct": 2057, + "selected_both_wrong": 1 + }, + { + "budget": 0.2, + "threshold": -0.0010652542114257812, + "operator": "ge", + "actual_fraction": 0.19340878828229027, + "metric": 0.1780928651788194, + "rescued": 4, + "harmed": 99, + "missed_rescue": 1278, + "avoided_harm": 678, + "selected_both_correct": 2220, + "selected_both_wrong": 1 + }, + { + "budget": 0.21, + "threshold": -0.0011425018310546875, + "operator": "ge", + "actual_fraction": 0.20306258322237017, + "metric": 0.17829989216687975, + "rescued": 5, + "harmed": 108, + "missed_rescue": 1277, + "avoided_harm": 669, + "selected_both_correct": 2325, + "selected_both_wrong": 2 + }, + { + "budget": 0.22, + "threshold": -0.0011968612670898438, + "operator": "ge", + "actual_fraction": 0.20838881491344874, + "metric": 0.1779854683890517, + "rescued": 5, + "harmed": 112, + "missed_rescue": 1277, + "avoided_harm": 665, + "selected_both_correct": 2385, + "selected_both_wrong": 2 + }, + { + "budget": 0.23, + "threshold": -0.0012493133544921875, + "operator": "ge", + "actual_fraction": 0.21537949400798934, + "metric": 0.17917657728013692, + "rescued": 6, + "harmed": 118, + "missed_rescue": 1276, + "avoided_harm": 659, + "selected_both_correct": 2462, + "selected_both_wrong": 2 + }, + { + "budget": 0.24, + "threshold": -0.0013513565063476562, + "operator": "ge", + "actual_fraction": 0.23135818908122505, + "metric": 0.17803687432776102, + "rescued": 6, + "harmed": 133, + "missed_rescue": 1276, + "avoided_harm": 644, + "selected_both_correct": 2638, + "selected_both_wrong": 3 + }, + { + "budget": 0.25, + "threshold": -0.0015010833740234375, + "operator": "gt", + "actual_fraction": 0.24800266311584554, + "metric": 0.18026802885144627, + "rescued": 10, + "harmed": 153, + "missed_rescue": 1272, + "avoided_harm": 624, + "selected_both_correct": 2814, + "selected_both_wrong": 3 + }, + { + "budget": 0.26, + "threshold": -0.00157928466796875, + "operator": "ge", + "actual_fraction": 0.2589880159786951, + "metric": 0.18119827871024874, + "rescued": 11, + "harmed": 163, + "missed_rescue": 1271, + "avoided_harm": 614, + "selected_both_correct": 2935, + "selected_both_wrong": 3 + }, + { + "budget": 0.27, + "threshold": -0.0017137527465820312, + "operator": "gt", + "actual_fraction": 0.27430093209054596, + "metric": 0.17963665085316138, + "rescued": 11, + "harmed": 185, + "missed_rescue": 1271, + "avoided_harm": 592, + "selected_both_correct": 3097, + "selected_both_wrong": 3 + }, + { + "budget": 0.28, + "threshold": -0.0017614364624023438, + "operator": "ge", + "actual_fraction": 0.281624500665779, + "metric": 0.17924095839307458, + "rescued": 11, + "harmed": 191, + "missed_rescue": 1271, + "avoided_harm": 586, + "selected_both_correct": 3179, + "selected_both_wrong": 3 + }, + { + "budget": 0.29, + "threshold": -0.0018463134765625, + "operator": "ge", + "actual_fraction": 0.28761651131824234, + "metric": 0.1789157174618718, + "rescued": 11, + "harmed": 196, + "missed_rescue": 1271, + "avoided_harm": 581, + "selected_both_correct": 3246, + "selected_both_wrong": 3 + }, + { + "budget": 0.3, + "threshold": -0.001934051513671875, + "operator": "ge", + "actual_fraction": 0.2969374167776298, + "metric": 0.1787206845800119, + "rescued": 11, + "harmed": 199, + "missed_rescue": 1271, + "avoided_harm": 578, + "selected_both_correct": 3355, + "selected_both_wrong": 3 + }, + { + "budget": 0.31, + "threshold": -0.0020427703857421875, + "operator": "gt", + "actual_fraction": 0.30459387483355527, + "metric": 0.17813242189706796, + "rescued": 11, + "harmed": 207, + "missed_rescue": 1271, + "avoided_harm": 570, + "selected_both_correct": 3438, + "selected_both_wrong": 4 + }, + { + "budget": 0.32, + "threshold": -0.0021495819091796875, + "operator": "gt", + "actual_fraction": 0.3129161118508655, + "metric": 0.17772000842137498, + "rescued": 12, + "harmed": 218, + "missed_rescue": 1270, + "avoided_harm": 559, + "selected_both_correct": 3526, + "selected_both_wrong": 4 + }, + { + "budget": 0.33, + "threshold": -0.002208709716796875, + "operator": "ge", + "actual_fraction": 0.31990679094540614, + "metric": 0.17791259734473341, + "rescued": 13, + "harmed": 227, + "missed_rescue": 1269, + "avoided_harm": 550, + "selected_both_correct": 3600, + "selected_both_wrong": 4 + }, + { + "budget": 0.34, + "threshold": -0.0023326873779296875, + "operator": "ge", + "actual_fraction": 0.3322237017310253, + "metric": 0.1789653934599337, + "rescued": 15, + "harmed": 238, + "missed_rescue": 1267, + "avoided_harm": 539, + "selected_both_correct": 3735, + "selected_both_wrong": 4 + }, + { + "budget": 0.35, + "threshold": -0.0024623870849609375, + "operator": "gt", + "actual_fraction": 0.34054593874833555, + "metric": 0.18042682200794885, + "rescued": 16, + "harmed": 249, + "missed_rescue": 1266, + "avoided_harm": 528, + "selected_both_correct": 3822, + "selected_both_wrong": 5 + }, + { + "budget": 0.36, + "threshold": -0.0025615692138671875, + "operator": "ge", + "actual_fraction": 0.3488681757656458, + "metric": 0.17962877325912951, + "rescued": 16, + "harmed": 260, + "missed_rescue": 1266, + "avoided_harm": 517, + "selected_both_correct": 3910, + "selected_both_wrong": 6 + }, + { + "budget": 0.37, + "threshold": -0.0026721954345703125, + "operator": "gt", + "actual_fraction": 0.3581890812250333, + "metric": 0.18158304685510065, + "rescued": 18, + "harmed": 271, + "missed_rescue": 1264, + "avoided_harm": 506, + "selected_both_correct": 4008, + "selected_both_wrong": 7 + }, + { + "budget": 0.38, + "threshold": -0.0028228759765625, + "operator": "gt", + "actual_fraction": 0.37017310252996005, + "metric": 0.18069239723097547, + "rescued": 18, + "harmed": 285, + "missed_rescue": 1264, + "avoided_harm": 492, + "selected_both_correct": 4138, + "selected_both_wrong": 7 + }, + { + "budget": 0.39, + "threshold": -0.0029354095458984375, + "operator": "ge", + "actual_fraction": 0.37683089214380827, + "metric": 0.1812845803851709, + "rescued": 20, + "harmed": 292, + "missed_rescue": 1262, + "avoided_harm": 485, + "selected_both_correct": 4209, + "selected_both_wrong": 7 + }, + { + "budget": 0.4, + "threshold": -0.00307464599609375, + "operator": "ge", + "actual_fraction": 0.38348868175765644, + "metric": 0.1824781664689562, + "rescued": 21, + "harmed": 301, + "missed_rescue": 1261, + "avoided_harm": 476, + "selected_both_correct": 4279, + "selected_both_wrong": 7 + }, + { + "budget": 0.41, + "threshold": -0.0033626556396484375, + "operator": "ge", + "actual_fraction": 0.4021304926764314, + "metric": 0.18220594501399923, + "rescued": 23, + "harmed": 321, + "missed_rescue": 1259, + "avoided_harm": 456, + "selected_both_correct": 4480, + "selected_both_wrong": 8 + }, + { + "budget": 0.42, + "threshold": -0.0035381317138671875, + "operator": "ge", + "actual_fraction": 0.41311584553928093, + "metric": 0.18162498930772586, + "rescued": 23, + "harmed": 329, + "missed_rescue": 1259, + "avoided_harm": 448, + "selected_both_correct": 4603, + "selected_both_wrong": 9 + }, + { + "budget": 0.43, + "threshold": -0.003765106201171875, + "operator": "ge", + "actual_fraction": 0.42676431424766975, + "metric": 0.18066434967152634, + "rescued": 23, + "harmed": 351, + "missed_rescue": 1259, + "avoided_harm": 426, + "selected_both_correct": 4745, + "selected_both_wrong": 9 + }, + { + "budget": 0.44, + "threshold": -0.00392913818359375, + "operator": "ge", + "actual_fraction": 0.4324234354194407, + "metric": 0.1803913789289444, + "rescued": 23, + "harmed": 355, + "missed_rescue": 1259, + "avoided_harm": 422, + "selected_both_correct": 4809, + "selected_both_wrong": 9 + }, + { + "budget": 0.45, + "threshold": -0.004085540771484375, + "operator": "ge", + "actual_fraction": 0.44107856191744343, + "metric": 0.17959468902865128, + "rescued": 23, + "harmed": 368, + "missed_rescue": 1259, + "avoided_harm": 409, + "selected_both_correct": 4900, + "selected_both_wrong": 9 + }, + { + "budget": 0.46, + "threshold": -0.004364013671875, + "operator": "ge", + "actual_fraction": 0.45772303595206393, + "metric": 0.17863383514615866, + "rescued": 24, + "harmed": 387, + "missed_rescue": 1258, + "avoided_harm": 390, + "selected_both_correct": 5078, + "selected_both_wrong": 11 + }, + { + "budget": 0.47, + "threshold": -0.004680633544921875, + "operator": "gt", + "actual_fraction": 0.4717043941411451, + "metric": 0.17957308538379646, + "rescued": 26, + "harmed": 405, + "missed_rescue": 1256, + "avoided_harm": 372, + "selected_both_correct": 5225, + "selected_both_wrong": 12 + }, + { + "budget": 0.48, + "threshold": -0.00490570068359375, + "operator": "ge", + "actual_fraction": 0.4826897470039947, + "metric": 0.17910245145856155, + "rescued": 27, + "harmed": 417, + "missed_rescue": 1255, + "avoided_harm": 360, + "selected_both_correct": 5344, + "selected_both_wrong": 12 + }, + { + "budget": 0.49, + "threshold": -0.005039215087890625, + "operator": "ge", + "actual_fraction": 0.4896804260985353, + "metric": 0.17871833907402393, + "rescued": 29, + "harmed": 425, + "missed_rescue": 1253, + "avoided_harm": 352, + "selected_both_correct": 5416, + "selected_both_wrong": 14 + }, + { + "budget": 0.5, + "threshold": -0.005321502685546875, + "operator": "ge", + "actual_fraction": 0.5013315579227696, + "metric": 0.18015785955296285, + "rescued": 57, + "harmed": 431, + "missed_rescue": 1225, + "avoided_harm": 346, + "selected_both_correct": 5516, + "selected_both_wrong": 20 + }, + { + "budget": 0.51, + "threshold": -0.005619049072265625, + "operator": "ge", + "actual_fraction": 0.5116511318242344, + "metric": 0.1810878167493375, + "rescued": 76, + "harmed": 435, + "missed_rescue": 1206, + "avoided_harm": 342, + "selected_both_correct": 5606, + "selected_both_wrong": 31 + }, + { + "budget": 0.52, + "threshold": -0.00586700439453125, + "operator": "gt", + "actual_fraction": 0.5183089214380826, + "metric": 0.18177962418520882, + "rescued": 92, + "harmed": 439, + "missed_rescue": 1190, + "avoided_harm": 338, + "selected_both_correct": 5662, + "selected_both_wrong": 35 + }, + { + "budget": 0.53, + "threshold": -0.00605010986328125, + "operator": "ge", + "actual_fraction": 0.5252996005326231, + "metric": 0.18246323539519374, + "rescued": 109, + "harmed": 445, + "missed_rescue": 1173, + "avoided_harm": 332, + "selected_both_correct": 5719, + "selected_both_wrong": 39 + }, + { + "budget": 0.54, + "threshold": -0.006439208984375, + "operator": "gt", + "actual_fraction": 0.5396138482023968, + "metric": 0.1839876166906015, + "rescued": 150, + "harmed": 461, + "missed_rescue": 1132, + "avoided_harm": 316, + "selected_both_correct": 5829, + "selected_both_wrong": 44 + }, + { + "budget": 0.55, + "threshold": -0.006641387939453125, + "operator": "ge", + "actual_fraction": 0.5452729693741678, + "metric": 0.1846674213907964, + "rescued": 164, + "harmed": 464, + "missed_rescue": 1118, + "avoided_harm": 313, + "selected_both_correct": 5875, + "selected_both_wrong": 49 + }, + { + "budget": 0.56, + "threshold": -0.0070648193359375, + "operator": "ge", + "actual_fraction": 0.559254327563249, + "metric": 0.18676294233791663, + "rescued": 196, + "harmed": 473, + "missed_rescue": 1086, + "avoided_harm": 304, + "selected_both_correct": 5990, + "selected_both_wrong": 61 + }, + { + "budget": 0.57, + "threshold": -0.007518768310546875, + "operator": "ge", + "actual_fraction": 0.572237017310253, + "metric": 0.18938663221175195, + "rescued": 230, + "harmed": 483, + "missed_rescue": 1052, + "avoided_harm": 294, + "selected_both_correct": 6093, + "selected_both_wrong": 70 + }, + { + "budget": 0.58, + "threshold": -0.00818634033203125, + "operator": "ge", + "actual_fraction": 0.5862183754993342, + "metric": 0.1909571002676625, + "rescued": 267, + "harmed": 496, + "missed_rescue": 1015, + "avoided_harm": 281, + "selected_both_correct": 6208, + "selected_both_wrong": 73 + }, + { + "budget": 0.59, + "threshold": -0.00878143310546875, + "operator": "gt", + "actual_fraction": 0.5998668442077231, + "metric": 0.1934300101790686, + "rescued": 299, + "harmed": 503, + "missed_rescue": 983, + "avoided_harm": 274, + "selected_both_correct": 6315, + "selected_both_wrong": 91 + }, + { + "budget": 0.6, + "threshold": -0.00955963134765625, + "operator": "gt", + "actual_fraction": 0.6191744340878829, + "metric": 0.1973701682130997, + "rescued": 350, + "harmed": 520, + "missed_rescue": 932, + "avoided_harm": 257, + "selected_both_correct": 6465, + "selected_both_wrong": 105 + }, + { + "budget": 0.61, + "threshold": -0.0099334716796875, + "operator": "ge", + "actual_fraction": 0.6264980026631158, + "metric": 0.1986727924998457, + "rescued": 370, + "harmed": 523, + "missed_rescue": 912, + "avoided_harm": 254, + "selected_both_correct": 6527, + "selected_both_wrong": 108 + }, + { + "budget": 0.62, + "threshold": -0.010650634765625, + "operator": "gt", + "actual_fraction": 0.6384820239680427, + "metric": 0.20186289191698192, + "rescued": 402, + "harmed": 532, + "missed_rescue": 880, + "avoided_harm": 245, + "selected_both_correct": 6621, + "selected_both_wrong": 117 + }, + { + "budget": 0.63, + "threshold": -0.01129150390625, + "operator": "ge", + "actual_fraction": 0.6461384820239681, + "metric": 0.20246031058520178, + "rescued": 425, + "harmed": 537, + "missed_rescue": 857, + "avoided_harm": 240, + "selected_both_correct": 6682, + "selected_both_wrong": 120 + }, + { + "budget": 0.64, + "threshold": -0.011871337890625, + "operator": "ge", + "actual_fraction": 0.653462050599201, + "metric": 0.20349944137622245, + "rescued": 444, + "harmed": 539, + "missed_rescue": 838, + "avoided_harm": 238, + "selected_both_correct": 6744, + "selected_both_wrong": 125 + }, + { + "budget": 0.65, + "threshold": -0.0125274658203125, + "operator": "gt", + "actual_fraction": 0.6597869507323568, + "metric": 0.20442990090722246, + "rescued": 461, + "harmed": 545, + "missed_rescue": 821, + "avoided_harm": 232, + "selected_both_correct": 6792, + "selected_both_wrong": 130 + }, + { + "budget": 0.66, + "threshold": -0.012969970703125, + "operator": "gt", + "actual_fraction": 0.6661118508655126, + "metric": 0.2044993942500819, + "rescued": 480, + "harmed": 556, + "missed_rescue": 802, + "avoided_harm": 221, + "selected_both_correct": 6834, + "selected_both_wrong": 134 + }, + { + "budget": 0.67, + "threshold": -0.01363372802734375, + "operator": "ge", + "actual_fraction": 0.6734354194407457, + "metric": 0.2064379542733177, + "rescued": 500, + "harmed": 561, + "missed_rescue": 782, + "avoided_harm": 216, + "selected_both_correct": 6889, + "selected_both_wrong": 142 + }, + { + "budget": 0.68, + "threshold": -0.01473236083984375, + "operator": "ge", + "actual_fraction": 0.6820905459387483, + "metric": 0.2065831206227621, + "rescued": 523, + "harmed": 566, + "missed_rescue": 759, + "avoided_harm": 211, + "selected_both_correct": 6958, + "selected_both_wrong": 149 + }, + { + "budget": 0.69, + "threshold": -0.015716552734375, + "operator": "ge", + "actual_fraction": 0.6927430093209055, + "metric": 0.20802802394105854, + "rescued": 553, + "harmed": 577, + "missed_rescue": 729, + "avoided_harm": 200, + "selected_both_correct": 7035, + "selected_both_wrong": 159 + }, + { + "budget": 0.7, + "threshold": -0.017578125, + "operator": "gt", + "actual_fraction": 0.7090545938748336, + "metric": 0.21169680596278434, + "rescued": 602, + "harmed": 584, + "missed_rescue": 680, + "avoided_harm": 193, + "selected_both_correct": 7159, + "selected_both_wrong": 175 + }, + { + "budget": 0.71, + "threshold": -0.0189056396484375, + "operator": "ge", + "actual_fraction": 0.7170439414114514, + "metric": 0.2139280527103987, + "rescued": 625, + "harmed": 588, + "missed_rescue": 657, + "avoided_harm": 189, + "selected_both_correct": 7220, + "selected_both_wrong": 183 + }, + { + "budget": 0.72, + "threshold": -0.019866943359375, + "operator": "ge", + "actual_fraction": 0.7243675099866844, + "metric": 0.2146900231025185, + "rescued": 647, + "harmed": 594, + "missed_rescue": 635, + "avoided_harm": 183, + "selected_both_correct": 7268, + "selected_both_wrong": 195 + }, + { + "budget": 0.73, + "threshold": -0.020965576171875, + "operator": "ge", + "actual_fraction": 0.7330226364846871, + "metric": 0.21653561577692484, + "rescued": 670, + "harmed": 599, + "missed_rescue": 612, + "avoided_harm": 178, + "selected_both_correct": 7338, + "selected_both_wrong": 201 + }, + { + "budget": 0.74, + "threshold": -0.021453857421875, + "operator": "ge", + "actual_fraction": 0.7363515312916112, + "metric": 0.21721689648133474, + "rescued": 681, + "harmed": 601, + "missed_rescue": 601, + "avoided_harm": 176, + "selected_both_correct": 7365, + "selected_both_wrong": 201 + }, + { + "budget": 0.75, + "threshold": -0.0248870849609375, + "operator": "ge", + "actual_fraction": 0.7486684420772304, + "metric": 0.22103445955201576, + "rescued": 718, + "harmed": 610, + "missed_rescue": 564, + "avoided_harm": 167, + "selected_both_correct": 7460, + "selected_both_wrong": 208 + }, + { + "budget": 0.76, + "threshold": -0.0269622802734375, + "operator": "ge", + "actual_fraction": 0.757323568575233, + "metric": 0.22340134248367294, + "rescued": 744, + "harmed": 618, + "missed_rescue": 538, + "avoided_harm": 159, + "selected_both_correct": 7522, + "selected_both_wrong": 216 + }, + { + "budget": 0.77, + "threshold": -0.0282135009765625, + "operator": "ge", + "actual_fraction": 0.7639813581890812, + "metric": 0.2269322568854726, + "rescued": 767, + "harmed": 620, + "missed_rescue": 515, + "avoided_harm": 157, + "selected_both_correct": 7571, + "selected_both_wrong": 222 + }, + { + "budget": 0.78, + "threshold": -0.0299835205078125, + "operator": "ge", + "actual_fraction": 0.7733022636484687, + "metric": 0.22757452798038888, + "rescued": 798, + "harmed": 628, + "missed_rescue": 484, + "avoided_harm": 149, + "selected_both_correct": 7636, + "selected_both_wrong": 230 + }, + { + "budget": 0.79, + "threshold": -0.033966064453125, + "operator": "ge", + "actual_fraction": 0.784953395472703, + "metric": 0.227821050912831, + "rescued": 824, + "harmed": 634, + "missed_rescue": 458, + "avoided_harm": 143, + "selected_both_correct": 7728, + "selected_both_wrong": 246 + }, + { + "budget": 0.8, + "threshold": -0.037200927734375, + "operator": "ge", + "actual_fraction": 0.7939414114513982, + "metric": 0.22991865671924006, + "rescued": 852, + "harmed": 642, + "missed_rescue": 430, + "avoided_harm": 135, + "selected_both_correct": 7793, + "selected_both_wrong": 253 + }, + { + "budget": 0.81, + "threshold": -0.04132080078125, + "operator": "ge", + "actual_fraction": 0.8039280958721704, + "metric": 0.23486191981046528, + "rescued": 884, + "harmed": 646, + "missed_rescue": 398, + "avoided_harm": 131, + "selected_both_correct": 7868, + "selected_both_wrong": 262 + }, + { + "budget": 0.82, + "threshold": -0.0455322265625, + "operator": "ge", + "actual_fraction": 0.8122503328894807, + "metric": 0.23803916047544685, + "rescued": 906, + "harmed": 655, + "missed_rescue": 376, + "avoided_harm": 122, + "selected_both_correct": 7924, + "selected_both_wrong": 275 + }, + { + "budget": 0.83, + "threshold": -0.051849365234375, + "operator": "ge", + "actual_fraction": 0.8215712383488681, + "metric": 0.24022947987632814, + "rescued": 933, + "harmed": 664, + "missed_rescue": 349, + "avoided_harm": 113, + "selected_both_correct": 7987, + "selected_both_wrong": 288 + }, + { + "budget": 0.84, + "threshold": -0.06878662109375, + "operator": "ge", + "actual_fraction": 0.8388814913448736, + "metric": 0.24528627405740036, + "rescued": 988, + "harmed": 672, + "missed_rescue": 294, + "avoided_harm": 105, + "selected_both_correct": 8116, + "selected_both_wrong": 304 + }, + { + "budget": 0.85, + "threshold": -0.08477783203125, + "operator": "ge", + "actual_fraction": 0.8535286284953395, + "metric": 0.249537136326177, + "rescued": 1028, + "harmed": 680, + "missed_rescue": 254, + "avoided_harm": 97, + "selected_both_correct": 8224, + "selected_both_wrong": 324 + }, + { + "budget": 0.86, + "threshold": -0.09912109375, + "operator": "ge", + "actual_fraction": 0.8638482023968043, + "metric": 0.25211074301800224, + "rescued": 1056, + "harmed": 688, + "missed_rescue": 226, + "avoided_harm": 89, + "selected_both_correct": 8299, + "selected_both_wrong": 337 + }, + { + "budget": 0.87, + "threshold": -0.1298828125, + "operator": "ge", + "actual_fraction": 0.8745006657789614, + "metric": 0.2540042317315216, + "rescued": 1076, + "harmed": 695, + "missed_rescue": 206, + "avoided_harm": 82, + "selected_both_correct": 8389, + "selected_both_wrong": 348 + }, + { + "budget": 0.88, + "threshold": -0.1441650390625, + "operator": "ge", + "actual_fraction": 0.8764980026631158, + "metric": 0.2540996517958595, + "rescued": 1080, + "harmed": 698, + "missed_rescue": 202, + "avoided_harm": 79, + "selected_both_correct": 8404, + "selected_both_wrong": 350 + }, + { + "budget": 0.89, + "threshold": -0.182373046875, + "operator": "ge", + "actual_fraction": 0.8841544607190412, + "metric": 0.25436194678023316, + "rescued": 1090, + "harmed": 705, + "missed_rescue": 192, + "avoided_harm": 72, + "selected_both_correct": 8468, + "selected_both_wrong": 361 + }, + { + "budget": 0.9, + "threshold": -0.2049560546875, + "operator": "ge", + "actual_fraction": 0.8891478029294274, + "metric": 0.2534339447803836, + "rescued": 1101, + "harmed": 709, + "missed_rescue": 181, + "avoided_harm": 68, + "selected_both_correct": 8506, + "selected_both_wrong": 368 + }, + { + "budget": 0.91, + "threshold": -0.25830078125, + "operator": "gt", + "actual_fraction": 0.8981358189081226, + "metric": 0.2563240880787248, + "rescued": 1126, + "harmed": 715, + "missed_rescue": 156, + "avoided_harm": 62, + "selected_both_correct": 8573, + "selected_both_wrong": 378 + }, + { + "budget": 0.92, + "threshold": -0.32080078125, + "operator": "ge", + "actual_fraction": 0.9054593874833555, + "metric": 0.25551845731687656, + "rescued": 1143, + "harmed": 721, + "missed_rescue": 139, + "avoided_harm": 56, + "selected_both_correct": 8632, + "selected_both_wrong": 384 + }, + { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9221038615179761, + "metric": 0.2597490004762105, + "rescued": 1180, + "harmed": 728, + "missed_rescue": 102, + "avoided_harm": 49, + "selected_both_correct": 8774, + "selected_both_wrong": 398 + }, + { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9254327563249002, + "metric": 0.26306323674307786, + "rescued": 1189, + "harmed": 729, + "missed_rescue": 93, + "avoided_harm": 48, + "selected_both_correct": 8800, + "selected_both_wrong": 402 + }, + { + "budget": 0.95, + "threshold": -0.611328125, + "operator": "ge", + "actual_fraction": 0.9364181091877497, + "metric": 0.26218185491449153, + "rescued": 1203, + "harmed": 738, + "missed_rescue": 79, + "avoided_harm": 39, + "selected_both_correct": 8897, + "selected_both_wrong": 414 + }, + { + "budget": 0.96, + "threshold": -0.70068359375, + "operator": "ge", + "actual_fraction": 0.953728362183755, + "metric": 0.26357052137555514, + "rescued": 1223, + "harmed": 754, + "missed_rescue": 59, + "avoided_harm": 23, + "selected_both_correct": 9055, + "selected_both_wrong": 428 + }, + { + "budget": 0.97, + "threshold": -0.75048828125, + "operator": "ge", + "actual_fraction": 0.96271637816245, + "metric": 0.26400290307863034, + "rescued": 1231, + "harmed": 758, + "missed_rescue": 51, + "avoided_harm": 19, + "selected_both_correct": 9150, + "selected_both_wrong": 429 + }, + { + "budget": 0.98, + "threshold": -0.7841796875, + "operator": "ge", + "actual_fraction": 0.9700399467376831, + "metric": 0.2647475724296899, + "rescued": 1237, + "harmed": 760, + "missed_rescue": 45, + "avoided_harm": 17, + "selected_both_correct": 9227, + "selected_both_wrong": 432 + }, + { + "budget": 0.99, + "threshold": -0.85107421875, + "operator": "ge", + "actual_fraction": 0.987017310252996, + "metric": 0.2649306125590882, + "rescued": 1257, + "harmed": 772, + "missed_rescue": 25, + "avoided_harm": 5, + "selected_both_correct": 9387, + "selected_both_wrong": 444 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.27001009067886245, + "rescued": 1282, + "harmed": 777, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9508, + "selected_both_wrong": 449 + } + ] + }, + "learned": { + "selected_budget_index": 50, + "validation_area_0_25": 0.23466194499464404, + "validation_point": { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.33126660689663334, + "rescued": 304, + "harmed": 95, + "missed_rescue": 3, + "avoided_harm": 125, + "selected_both_correct": 916, + "selected_both_wrong": 85 + }, + "test_point": { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4876830892143808, + "metric": 0.29875800068449215, + "rescued": 1249, + "harmed": 337, + "missed_rescue": 33, + "avoided_harm": 440, + "selected_both_correct": 3843, + "selected_both_wrong": 431 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.3274500370025635, + "operator": "ge", + "actual_fraction": 0.009985734664764621, + "metric": 0.18570977475658457, + "rescued": 12, + "harmed": 1, + "missed_rescue": 295, + "avoided_harm": 219, + "selected_both_correct": 12, + "selected_both_wrong": 3 + }, + { + "budget": 0.02, + "threshold": 2.223353862762451, + "operator": "ge", + "actual_fraction": 0.019971469329529243, + "metric": 0.19072678038582708, + "rescued": 24, + "harmed": 1, + "missed_rescue": 283, + "avoided_harm": 219, + "selected_both_correct": 26, + "selected_both_wrong": 5 + }, + { + "budget": 0.03, + "threshold": 2.105621099472046, + "operator": "ge", + "actual_fraction": 0.029957203994293864, + "metric": 0.1947361119928147, + "rescued": 33, + "harmed": 1, + "missed_rescue": 274, + "avoided_harm": 219, + "selected_both_correct": 40, + "selected_both_wrong": 10 + }, + { + "budget": 0.04, + "threshold": 1.9981958866119385, + "operator": "ge", + "actual_fraction": 0.039942938659058486, + "metric": 0.20027934358652552, + "rescued": 44, + "harmed": 1, + "missed_rescue": 263, + "avoided_harm": 219, + "selected_both_correct": 52, + "selected_both_wrong": 15 + }, + { + "budget": 0.05, + "threshold": 1.949358582496643, + "operator": "ge", + "actual_fraction": 0.04992867332382311, + "metric": 0.21085585634765963, + "rescued": 57, + "harmed": 1, + "missed_rescue": 250, + "avoided_harm": 219, + "selected_both_correct": 65, + "selected_both_wrong": 17 + }, + { + "budget": 0.06, + "threshold": 1.8504760265350342, + "operator": "ge", + "actual_fraction": 0.05991440798858773, + "metric": 0.2155935788086818, + "rescued": 68, + "harmed": 2, + "missed_rescue": 239, + "avoided_harm": 218, + "selected_both_correct": 78, + "selected_both_wrong": 20 + }, + { + "budget": 0.07, + "threshold": 1.6561405658721924, + "operator": "ge", + "actual_fraction": 0.06990014265335236, + "metric": 0.2201008442387753, + "rescued": 76, + "harmed": 4, + "missed_rescue": 231, + "avoided_harm": 216, + "selected_both_correct": 93, + "selected_both_wrong": 23 + }, + { + "budget": 0.08, + "threshold": 1.5329420566558838, + "operator": "ge", + "actual_fraction": 0.07988587731811697, + "metric": 0.22831044506816361, + "rescued": 90, + "harmed": 5, + "missed_rescue": 217, + "avoided_harm": 215, + "selected_both_correct": 106, + "selected_both_wrong": 23 + }, + { + "budget": 0.09, + "threshold": 1.380273461341858, + "operator": "ge", + "actual_fraction": 0.0898716119828816, + "metric": 0.23143136333139114, + "rescued": 96, + "harmed": 6, + "missed_rescue": 211, + "avoided_harm": 214, + "selected_both_correct": 124, + "selected_both_wrong": 26 + }, + { + "budget": 0.1, + "threshold": 1.32539701461792, + "operator": "ge", + "actual_fraction": 0.09985734664764621, + "metric": 0.23760484070252644, + "rescued": 104, + "harmed": 8, + "missed_rescue": 203, + "avoided_harm": 212, + "selected_both_correct": 140, + "selected_both_wrong": 28 + }, + { + "budget": 0.11, + "threshold": 1.2173664569854736, + "operator": "ge", + "actual_fraction": 0.10984308131241084, + "metric": 0.23824320355020984, + "rescued": 110, + "harmed": 9, + "missed_rescue": 197, + "avoided_harm": 211, + "selected_both_correct": 161, + "selected_both_wrong": 28 + }, + { + "budget": 0.12, + "threshold": 1.1719112396240234, + "operator": "ge", + "actual_fraction": 0.11982881597717546, + "metric": 0.2407100587309389, + "rescued": 115, + "harmed": 9, + "missed_rescue": 192, + "avoided_harm": 211, + "selected_both_correct": 181, + "selected_both_wrong": 31 + }, + { + "budget": 0.13, + "threshold": 1.1186922788619995, + "operator": "ge", + "actual_fraction": 0.12981455064194009, + "metric": 0.24134727061556333, + "rescued": 121, + "harmed": 11, + "missed_rescue": 186, + "avoided_harm": 209, + "selected_both_correct": 199, + "selected_both_wrong": 33 + }, + { + "budget": 0.14, + "threshold": 1.0748379230499268, + "operator": "ge", + "actual_fraction": 0.1398002853067047, + "metric": 0.24423106111883697, + "rescued": 128, + "harmed": 16, + "missed_rescue": 179, + "avoided_harm": 204, + "selected_both_correct": 213, + "selected_both_wrong": 35 + }, + { + "budget": 0.15, + "threshold": 1.0427184104919434, + "operator": "ge", + "actual_fraction": 0.14978601997146934, + "metric": 0.24892068530842207, + "rescued": 135, + "harmed": 16, + "missed_rescue": 172, + "avoided_harm": 204, + "selected_both_correct": 234, + "selected_both_wrong": 35 + }, + { + "budget": 0.16, + "threshold": 1.0111123323440552, + "operator": "ge", + "actual_fraction": 0.15977175463623394, + "metric": 0.252253667728857, + "rescued": 141, + "harmed": 16, + "missed_rescue": 166, + "avoided_harm": 204, + "selected_both_correct": 254, + "selected_both_wrong": 37 + }, + { + "budget": 0.17, + "threshold": 0.9669641852378845, + "operator": "ge", + "actual_fraction": 0.16975748930099857, + "metric": 0.25275417712816833, + "rescued": 146, + "harmed": 18, + "missed_rescue": 161, + "avoided_harm": 202, + "selected_both_correct": 272, + "selected_both_wrong": 40 + }, + { + "budget": 0.18, + "threshold": 0.9490972757339478, + "operator": "ge", + "actual_fraction": 0.1797432239657632, + "metric": 0.25643535926629674, + "rescued": 152, + "harmed": 19, + "missed_rescue": 155, + "avoided_harm": 201, + "selected_both_correct": 292, + "selected_both_wrong": 41 + }, + { + "budget": 0.19, + "threshold": 0.9165292382240295, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.2576337529504327, + "rescued": 159, + "harmed": 22, + "missed_rescue": 148, + "avoided_harm": 198, + "selected_both_correct": 310, + "selected_both_wrong": 41 + }, + { + "budget": 0.2, + "threshold": 0.8978593349456787, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.2581122448979592, + "rescued": 164, + "harmed": 24, + "missed_rescue": 143, + "avoided_harm": 196, + "selected_both_correct": 330, + "selected_both_wrong": 42 + }, + { + "budget": 0.21, + "threshold": 0.854847252368927, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.2602060885029815, + "rescued": 169, + "harmed": 25, + "missed_rescue": 138, + "avoided_harm": 195, + "selected_both_correct": 350, + "selected_both_wrong": 44 + }, + { + "budget": 0.22, + "threshold": 0.8254434466362, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.2635325636000918, + "rescued": 174, + "harmed": 25, + "missed_rescue": 133, + "avoided_harm": 195, + "selected_both_correct": 371, + "selected_both_wrong": 46 + }, + { + "budget": 0.23, + "threshold": 0.8046320080757141, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.24799578059071728, + "rescued": 178, + "harmed": 29, + "missed_rescue": 129, + "avoided_harm": 191, + "selected_both_correct": 391, + "selected_both_wrong": 46 + }, + { + "budget": 0.24, + "threshold": 0.7761231660842896, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.25061334511913563, + "rescued": 184, + "harmed": 31, + "missed_rescue": 123, + "avoided_harm": 189, + "selected_both_correct": 410, + "selected_both_wrong": 47 + }, + { + "budget": 0.25, + "threshold": 0.7452236413955688, + "operator": "ge", + "actual_fraction": 0.24964336661911554, + "metric": 0.25309147374364765, + "rescued": 190, + "harmed": 32, + "missed_rescue": 117, + "avoided_harm": 188, + "selected_both_correct": 430, + "selected_both_wrong": 48 + }, + { + "budget": 0.26, + "threshold": 0.7167489528656006, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.25499133574730426, + "rescued": 196, + "harmed": 34, + "missed_rescue": 111, + "avoided_harm": 186, + "selected_both_correct": 448, + "selected_both_wrong": 50 + }, + { + "budget": 0.27, + "threshold": 0.6837625503540039, + "operator": "ge", + "actual_fraction": 0.26961483594864477, + "metric": 0.25870776744375423, + "rescued": 201, + "harmed": 34, + "missed_rescue": 106, + "avoided_harm": 186, + "selected_both_correct": 468, + "selected_both_wrong": 53 + }, + { + "budget": 0.28, + "threshold": 0.633089005947113, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.25965649053463696, + "rescued": 205, + "harmed": 36, + "missed_rescue": 102, + "avoided_harm": 184, + "selected_both_correct": 488, + "selected_both_wrong": 55 + }, + { + "budget": 0.29, + "threshold": 0.5875619649887085, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.2629594041536389, + "rescued": 213, + "harmed": 39, + "missed_rescue": 94, + "avoided_harm": 181, + "selected_both_correct": 505, + "selected_both_wrong": 55 + }, + { + "budget": 0.3, + "threshold": 0.5604347586631775, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.26353457364905875, + "rescued": 219, + "harmed": 45, + "missed_rescue": 88, + "avoided_harm": 175, + "selected_both_correct": 521, + "selected_both_wrong": 55 + }, + { + "budget": 0.31, + "threshold": 0.5320743918418884, + "operator": "ge", + "actual_fraction": 0.3095577746077033, + "metric": 0.26608946608946615, + "rescued": 225, + "harmed": 47, + "missed_rescue": 82, + "avoided_harm": 173, + "selected_both_correct": 540, + "selected_both_wrong": 56 + }, + { + "budget": 0.32, + "threshold": 0.49417412281036377, + "operator": "ge", + "actual_fraction": 0.3195435092724679, + "metric": 0.2673115302724549, + "rescued": 230, + "harmed": 51, + "missed_rescue": 77, + "avoided_harm": 169, + "selected_both_correct": 558, + "selected_both_wrong": 57 + }, + { + "budget": 0.33, + "threshold": 0.46743881702423096, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.26933684672815106, + "rescued": 236, + "harmed": 53, + "missed_rescue": 71, + "avoided_harm": 167, + "selected_both_correct": 577, + "selected_both_wrong": 58 + }, + { + "budget": 0.34, + "threshold": 0.4429023861885071, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.27352180339985216, + "rescued": 243, + "harmed": 54, + "missed_rescue": 64, + "avoided_harm": 166, + "selected_both_correct": 597, + "selected_both_wrong": 58 + }, + { + "budget": 0.35, + "threshold": 0.40694165229797363, + "operator": "ge", + "actual_fraction": 0.34950071326676174, + "metric": 0.275880713431076, + "rescued": 247, + "harmed": 60, + "missed_rescue": 60, + "avoided_harm": 160, + "selected_both_correct": 612, + "selected_both_wrong": 61 + }, + { + "budget": 0.36, + "threshold": 0.3611782193183899, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.27761867831050957, + "rescued": 253, + "harmed": 64, + "missed_rescue": 54, + "avoided_harm": 156, + "selected_both_correct": 629, + "selected_both_wrong": 62 + }, + { + "budget": 0.37, + "threshold": 0.3311936855316162, + "operator": "ge", + "actual_fraction": 0.369472182596291, + "metric": 0.28017336393459635, + "rescued": 257, + "harmed": 66, + "missed_rescue": 50, + "avoided_harm": 154, + "selected_both_correct": 650, + "selected_both_wrong": 63 + }, + { + "budget": 0.38, + "threshold": 0.28899651765823364, + "operator": "ge", + "actual_fraction": 0.37945791726105566, + "metric": 0.2825320512820513, + "rescued": 263, + "harmed": 70, + "missed_rescue": 44, + "avoided_harm": 150, + "selected_both_correct": 667, + "selected_both_wrong": 64 + }, + { + "budget": 0.39, + "threshold": 0.19986218214035034, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.28580355455355455, + "rescued": 267, + "harmed": 71, + "missed_rescue": 40, + "avoided_harm": 149, + "selected_both_correct": 689, + "selected_both_wrong": 65 + }, + { + "budget": 0.4, + "threshold": 0.157886803150177, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.3015271169266015, + "rescued": 271, + "harmed": 72, + "missed_rescue": 36, + "avoided_harm": 148, + "selected_both_correct": 709, + "selected_both_wrong": 68 + }, + { + "budget": 0.41, + "threshold": 0.12043046951293945, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.30656010656010657, + "rescued": 275, + "harmed": 74, + "missed_rescue": 32, + "avoided_harm": 146, + "selected_both_correct": 730, + "selected_both_wrong": 69 + }, + { + "budget": 0.42, + "threshold": 0.06623327732086182, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.30573593073593075, + "rescued": 277, + "harmed": 77, + "missed_rescue": 30, + "avoided_harm": 143, + "selected_both_correct": 750, + "selected_both_wrong": 72 + }, + { + "budget": 0.43, + "threshold": -0.01827472448348999, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.3122803645750104, + "rescued": 283, + "harmed": 79, + "missed_rescue": 24, + "avoided_harm": 141, + "selected_both_correct": 770, + "selected_both_wrong": 72 + }, + { + "budget": 0.44, + "threshold": -0.14177680015563965, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.32014301599541106, + "rescued": 290, + "harmed": 81, + "missed_rescue": 17, + "avoided_harm": 139, + "selected_both_correct": 788, + "selected_both_wrong": 73 + }, + { + "budget": 0.45, + "threshold": -0.270923376083374, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.3274352926559858, + "rescued": 295, + "harmed": 82, + "missed_rescue": 12, + "avoided_harm": 138, + "selected_both_correct": 808, + "selected_both_wrong": 75 + }, + { + "budget": 0.46, + "threshold": -0.4213377833366394, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.3269493177387914, + "rescued": 299, + "harmed": 85, + "missed_rescue": 8, + "avoided_harm": 135, + "selected_both_correct": 826, + "selected_both_wrong": 78 + }, + { + "budget": 0.47, + "threshold": -0.6602996587753296, + "operator": "ge", + "actual_fraction": 0.4693295292439372, + "metric": 0.3206497235233282, + "rescued": 299, + "harmed": 88, + "missed_rescue": 8, + "avoided_harm": 132, + "selected_both_correct": 849, + "selected_both_wrong": 80 + }, + { + "budget": 0.48, + "threshold": -0.8612939715385437, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.3201660892787428, + "rescued": 300, + "harmed": 89, + "missed_rescue": 7, + "avoided_harm": 131, + "selected_both_correct": 873, + "selected_both_wrong": 82 + }, + { + "budget": 0.49, + "threshold": -0.9715782999992371, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.325869430652523, + "rescued": 303, + "harmed": 91, + "missed_rescue": 4, + "avoided_harm": 129, + "selected_both_correct": 894, + "selected_both_wrong": 84 + }, + { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.33126660689663334, + "rescued": 304, + "harmed": 95, + "missed_rescue": 3, + "avoided_harm": 125, + "selected_both_correct": 916, + "selected_both_wrong": 85 + }, + { + "budget": 0.51, + "threshold": -1.6505446434020996, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.3264586577163139, + "rescued": 304, + "harmed": 98, + "missed_rescue": 3, + "avoided_harm": 122, + "selected_both_correct": 941, + "selected_both_wrong": 85 + }, + { + "budget": 0.52, + "threshold": -2.6140921115875244, + "operator": "ge", + "actual_fraction": 0.5192582025677603, + "metric": 0.3264586577163139, + "rescued": 304, + "harmed": 98, + "missed_rescue": 3, + "avoided_harm": 122, + "selected_both_correct": 968, + "selected_both_wrong": 86 + }, + { + "budget": 0.53, + "threshold": -2.6907107830047607, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.3235189795049722, + "rescued": 304, + "harmed": 102, + "missed_rescue": 3, + "avoided_harm": 118, + "selected_both_correct": 992, + "selected_both_wrong": 86 + }, + { + "budget": 0.54, + "threshold": -2.8344802856445312, + "operator": "ge", + "actual_fraction": 0.5392296718972895, + "metric": 0.3235189795049722, + "rescued": 304, + "harmed": 102, + "missed_rescue": 3, + "avoided_harm": 118, + "selected_both_correct": 1020, + "selected_both_wrong": 86 + }, + { + "budget": 0.55, + "threshold": -2.905412197113037, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.3203578942095772, + "rescued": 304, + "harmed": 106, + "missed_rescue": 3, + "avoided_harm": 114, + "selected_both_correct": 1044, + "selected_both_wrong": 86 + }, + { + "budget": 0.56, + "threshold": -2.9482638835906982, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.31783773291925466, + "rescued": 304, + "harmed": 107, + "missed_rescue": 3, + "avoided_harm": 113, + "selected_both_correct": 1071, + "selected_both_wrong": 86 + }, + { + "budget": 0.57, + "threshold": -2.9957408905029297, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.31989933342152455, + "rescued": 305, + "harmed": 111, + "missed_rescue": 2, + "avoided_harm": 109, + "selected_both_correct": 1094, + "selected_both_wrong": 86 + }, + { + "budget": 0.58, + "threshold": -3.1037726402282715, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.31989933342152455, + "rescued": 305, + "harmed": 111, + "missed_rescue": 2, + "avoided_harm": 109, + "selected_both_correct": 1122, + "selected_both_wrong": 86 + }, + { + "budget": 0.59, + "threshold": -3.1757895946502686, + "operator": "ge", + "actual_fraction": 0.5891583452211127, + "metric": 0.3184090005781182, + "rescued": 305, + "harmed": 113, + "missed_rescue": 2, + "avoided_harm": 107, + "selected_both_correct": 1148, + "selected_both_wrong": 86 + }, + { + "budget": 0.6, + "threshold": -3.253206729888916, + "operator": "ge", + "actual_fraction": 0.5991440798858774, + "metric": 0.3177705940408353, + "rescued": 305, + "harmed": 114, + "missed_rescue": 2, + "avoided_harm": 106, + "selected_both_correct": 1175, + "selected_both_wrong": 86 + }, + { + "budget": 0.61, + "threshold": -3.2825026512145996, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.31694234643839414, + "rescued": 305, + "harmed": 115, + "missed_rescue": 2, + "avoided_harm": 105, + "selected_both_correct": 1202, + "selected_both_wrong": 86 + }, + { + "budget": 0.62, + "threshold": -3.322240114212036, + "operator": "ge", + "actual_fraction": 0.6191155492154066, + "metric": 0.3111129112312657, + "rescued": 305, + "harmed": 121, + "missed_rescue": 2, + "avoided_harm": 99, + "selected_both_correct": 1224, + "selected_both_wrong": 86 + }, + { + "budget": 0.63, + "threshold": -3.3503003120422363, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.30995657450601133, + "rescued": 305, + "harmed": 123, + "missed_rescue": 2, + "avoided_harm": 97, + "selected_both_correct": 1250, + "selected_both_wrong": 86 + }, + { + "budget": 0.64, + "threshold": -3.377089738845825, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.3083114768968865, + "rescued": 305, + "harmed": 126, + "missed_rescue": 2, + "avoided_harm": 94, + "selected_both_correct": 1275, + "selected_both_wrong": 86 + }, + { + "budget": 0.65, + "threshold": -3.4055864810943604, + "operator": "ge", + "actual_fraction": 0.6490727532097005, + "metric": 0.30666904687523244, + "rescued": 305, + "harmed": 128, + "missed_rescue": 2, + "avoided_harm": 92, + "selected_both_correct": 1301, + "selected_both_wrong": 86 + }, + { + "budget": 0.66, + "threshold": -3.430248498916626, + "operator": "ge", + "actual_fraction": 0.6590584878744651, + "metric": 0.30339954163483573, + "rescued": 305, + "harmed": 131, + "missed_rescue": 2, + "avoided_harm": 89, + "selected_both_correct": 1326, + "selected_both_wrong": 86 + }, + { + "budget": 0.67, + "threshold": -3.4584414958953857, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.3011204481792717, + "rescued": 305, + "harmed": 135, + "missed_rescue": 2, + "avoided_harm": 85, + "selected_both_correct": 1350, + "selected_both_wrong": 86 + }, + { + "budget": 0.68, + "threshold": -3.502922773361206, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.2968337073049115, + "rescued": 305, + "harmed": 140, + "missed_rescue": 2, + "avoided_harm": 80, + "selected_both_correct": 1373, + "selected_both_wrong": 86 + }, + { + "budget": 0.69, + "threshold": -3.532865285873413, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.2968337073049115, + "rescued": 305, + "harmed": 140, + "missed_rescue": 2, + "avoided_harm": 80, + "selected_both_correct": 1401, + "selected_both_wrong": 86 + }, + { + "budget": 0.7, + "threshold": -3.5550942420959473, + "operator": "ge", + "actual_fraction": 0.6990014265335235, + "metric": 0.2951499808548299, + "rescued": 305, + "harmed": 143, + "missed_rescue": 2, + "avoided_harm": 77, + "selected_both_correct": 1426, + "selected_both_wrong": 86 + }, + { + "budget": 0.71, + "threshold": -3.566344976425171, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.2985697751322751, + "rescued": 306, + "harmed": 144, + "missed_rescue": 1, + "avoided_harm": 76, + "selected_both_correct": 1452, + "selected_both_wrong": 86 + }, + { + "budget": 0.72, + "threshold": -3.6056067943573, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.29288998820248824, + "rescued": 306, + "harmed": 150, + "missed_rescue": 1, + "avoided_harm": 70, + "selected_both_correct": 1474, + "selected_both_wrong": 86 + }, + { + "budget": 0.73, + "threshold": -3.628530979156494, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.2916863054994102, + "rescued": 306, + "harmed": 156, + "missed_rescue": 1, + "avoided_harm": 64, + "selected_both_correct": 1496, + "selected_both_wrong": 86 + }, + { + "budget": 0.74, + "threshold": -3.6580705642700195, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.2908398260842505, + "rescued": 306, + "harmed": 158, + "missed_rescue": 1, + "avoided_harm": 62, + "selected_both_correct": 1521, + "selected_both_wrong": 87 + }, + { + "budget": 0.75, + "threshold": -3.6900978088378906, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.29144523554686896, + "rescued": 307, + "harmed": 161, + "missed_rescue": 0, + "avoided_harm": 59, + "selected_both_correct": 1545, + "selected_both_wrong": 87 + }, + { + "budget": 0.76, + "threshold": -3.7050747871398926, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.28771367521367525, + "rescued": 307, + "harmed": 164, + "missed_rescue": 0, + "avoided_harm": 56, + "selected_both_correct": 1569, + "selected_both_wrong": 88 + }, + { + "budget": 0.77, + "threshold": -3.720116138458252, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.28342343829529665, + "rescued": 307, + "harmed": 170, + "missed_rescue": 0, + "avoided_harm": 50, + "selected_both_correct": 1591, + "selected_both_wrong": 88 + }, + { + "budget": 0.78, + "threshold": -3.7399919033050537, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.27967055835908294, + "rescued": 307, + "harmed": 175, + "missed_rescue": 0, + "avoided_harm": 45, + "selected_both_correct": 1614, + "selected_both_wrong": 88 + }, + { + "budget": 0.79, + "threshold": -3.799569606781006, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.2754483860214125, + "rescued": 307, + "harmed": 180, + "missed_rescue": 0, + "avoided_harm": 40, + "selected_both_correct": 1637, + "selected_both_wrong": 88 + }, + { + "budget": 0.8, + "threshold": -3.8980000019073486, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.2750612892472189, + "rescued": 307, + "harmed": 181, + "missed_rescue": 0, + "avoided_harm": 39, + "selected_both_correct": 1664, + "selected_both_wrong": 88 + }, + { + "budget": 0.81, + "threshold": -4.0607757568359375, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.27434520884520885, + "rescued": 307, + "harmed": 186, + "missed_rescue": 0, + "avoided_harm": 34, + "selected_both_correct": 1687, + "selected_both_wrong": 88 + }, + { + "budget": 0.82, + "threshold": -8.964031219482422, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.27434520884520885, + "rescued": 307, + "harmed": 186, + "missed_rescue": 0, + "avoided_harm": 34, + "selected_both_correct": 1715, + "selected_both_wrong": 88 + }, + { + "budget": 0.83, + "threshold": -9.02132511138916, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.27363625362132826, + "rescued": 307, + "harmed": 187, + "missed_rescue": 0, + "avoided_harm": 33, + "selected_both_correct": 1742, + "selected_both_wrong": 88 + }, + { + "budget": 0.84, + "threshold": -9.09000015258789, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.2722230802027571, + "rescued": 307, + "harmed": 189, + "missed_rescue": 0, + "avoided_harm": 31, + "selected_both_correct": 1768, + "selected_both_wrong": 88 + }, + { + "budget": 0.85, + "threshold": -9.12938404083252, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.2722230802027571, + "rescued": 307, + "harmed": 189, + "missed_rescue": 0, + "avoided_harm": 31, + "selected_both_correct": 1796, + "selected_both_wrong": 88 + }, + { + "budget": 0.86, + "threshold": -9.17154312133789, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.2708398536448072, + "rescued": 307, + "harmed": 191, + "missed_rescue": 0, + "avoided_harm": 29, + "selected_both_correct": 1822, + "selected_both_wrong": 88 + }, + { + "budget": 0.87, + "threshold": -9.273024559020996, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.2708398536448072, + "rescued": 307, + "harmed": 191, + "missed_rescue": 0, + "avoided_harm": 29, + "selected_both_correct": 1850, + "selected_both_wrong": 88 + }, + { + "budget": 0.88, + "threshold": -9.444879531860352, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.27016508981214865, + "rescued": 307, + "harmed": 192, + "missed_rescue": 0, + "avoided_harm": 28, + "selected_both_correct": 1877, + "selected_both_wrong": 88 + }, + { + "budget": 0.89, + "threshold": -9.566991806030273, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.27016508981214865, + "rescued": 307, + "harmed": 192, + "missed_rescue": 0, + "avoided_harm": 28, + "selected_both_correct": 1905, + "selected_both_wrong": 88 + }, + { + "budget": 0.9, + "threshold": -9.675031661987305, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.27016508981214865, + "rescued": 307, + "harmed": 192, + "missed_rescue": 0, + "avoided_harm": 28, + "selected_both_correct": 1933, + "selected_both_wrong": 88 + }, + { + "budget": 0.91, + "threshold": -9.739017486572266, + "operator": "ge", + "actual_fraction": 0.9087018544935807, + "metric": 0.2694835977031099, + "rescued": 307, + "harmed": 193, + "missed_rescue": 0, + "avoided_harm": 27, + "selected_both_correct": 1960, + "selected_both_wrong": 88 + }, + { + "budget": 0.92, + "threshold": -9.787779808044434, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.2680528165105032, + "rescued": 307, + "harmed": 196, + "missed_rescue": 0, + "avoided_harm": 24, + "selected_both_correct": 1985, + "selected_both_wrong": 88 + }, + { + "budget": 0.93, + "threshold": -9.832305908203125, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2680528165105032, + "rescued": 307, + "harmed": 196, + "missed_rescue": 0, + "avoided_harm": 24, + "selected_both_correct": 2013, + "selected_both_wrong": 88 + }, + { + "budget": 0.94, + "threshold": -9.88642692565918, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.2673844613676863, + "rescued": 307, + "harmed": 201, + "missed_rescue": 0, + "avoided_harm": 19, + "selected_both_correct": 2036, + "selected_both_wrong": 88 + }, + { + "budget": 0.95, + "threshold": -9.976344108581543, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.26575274061687104, + "rescued": 307, + "harmed": 203, + "missed_rescue": 0, + "avoided_harm": 17, + "selected_both_correct": 2062, + "selected_both_wrong": 88 + }, + { + "budget": 0.96, + "threshold": -10.03441047668457, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.261865308604439, + "rescued": 307, + "harmed": 209, + "missed_rescue": 0, + "avoided_harm": 11, + "selected_both_correct": 2084, + "selected_both_wrong": 88 + }, + { + "budget": 0.97, + "threshold": -10.085777282714844, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.2589514652014652, + "rescued": 307, + "harmed": 212, + "missed_rescue": 0, + "avoided_harm": 8, + "selected_both_correct": 2109, + "selected_both_wrong": 88 + }, + { + "budget": 0.98, + "threshold": -10.1488676071167, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.25830826953457287, + "rescued": 307, + "harmed": 213, + "missed_rescue": 0, + "avoided_harm": 7, + "selected_both_correct": 2136, + "selected_both_wrong": 88 + }, + { + "budget": 0.99, + "threshold": -10.265552520751953, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.25668767695229905, + "rescued": 307, + "harmed": 216, + "missed_rescue": 0, + "avoided_harm": 4, + "selected_both_correct": 2161, + "selected_both_wrong": 88 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.25448626222520154, + "rescued": 307, + "harmed": 220, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 2189, + "selected_both_wrong": 88 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.3274500370025635, + "operator": "ge", + "actual_fraction": 0.013648468708388815, + "metric": 0.18693471712069915, + "rescued": 72, + "harmed": 3, + "missed_rescue": 1210, + "avoided_harm": 774, + "selected_both_correct": 69, + "selected_both_wrong": 20 + }, + { + "budget": 0.02, + "threshold": 2.223353862762451, + "operator": "ge", + "actual_fraction": 0.019640479360852198, + "metric": 0.18781313491575913, + "rescued": 98, + "harmed": 7, + "missed_rescue": 1184, + "avoided_harm": 770, + "selected_both_correct": 104, + "selected_both_wrong": 27 + }, + { + "budget": 0.03, + "threshold": 2.105621099472046, + "operator": "ge", + "actual_fraction": 0.031291611185086554, + "metric": 0.1913128749335621, + "rescued": 154, + "harmed": 10, + "missed_rescue": 1128, + "avoided_harm": 767, + "selected_both_correct": 170, + "selected_both_wrong": 42 + }, + { + "budget": 0.04, + "threshold": 1.9981958866119385, + "operator": "ge", + "actual_fraction": 0.04061251664447404, + "metric": 0.19266008228682668, + "rescued": 193, + "harmed": 16, + "missed_rescue": 1089, + "avoided_harm": 761, + "selected_both_correct": 219, + "selected_both_wrong": 60 + }, + { + "budget": 0.05, + "threshold": 1.949358582496643, + "operator": "ge", + "actual_fraction": 0.045938748335552594, + "metric": 0.19531128485766117, + "rescued": 224, + "harmed": 18, + "missed_rescue": 1058, + "avoided_harm": 759, + "selected_both_correct": 249, + "selected_both_wrong": 61 + }, + { + "budget": 0.06, + "threshold": 1.8504760265350342, + "operator": "ge", + "actual_fraction": 0.0559254327563249, + "metric": 0.19800731707634547, + "rescued": 268, + "harmed": 23, + "missed_rescue": 1014, + "avoided_harm": 754, + "selected_both_correct": 302, + "selected_both_wrong": 79 + }, + { + "budget": 0.07, + "threshold": 1.6561405658721924, + "operator": "ge", + "actual_fraction": 0.07456724367509987, + "metric": 0.19763843353437874, + "rescued": 336, + "harmed": 37, + "missed_rescue": 946, + "avoided_harm": 740, + "selected_both_correct": 409, + "selected_both_wrong": 114 + }, + { + "budget": 0.08, + "threshold": 1.5329420566558838, + "operator": "ge", + "actual_fraction": 0.08688415446071904, + "metric": 0.1974112776278713, + "rescued": 386, + "harmed": 48, + "missed_rescue": 896, + "avoided_harm": 729, + "selected_both_correct": 472, + "selected_both_wrong": 138 + }, + { + "budget": 0.09, + "threshold": 1.380273461341858, + "operator": "ge", + "actual_fraction": 0.09587217043941411, + "metric": 0.20030220967092682, + "rescued": 421, + "harmed": 54, + "missed_rescue": 861, + "avoided_harm": 723, + "selected_both_correct": 529, + "selected_both_wrong": 148 + }, + { + "budget": 0.1, + "threshold": 1.32539701461792, + "operator": "ge", + "actual_fraction": 0.09920106524633822, + "metric": 0.20211074868345033, + "rescued": 432, + "harmed": 56, + "missed_rescue": 850, + "avoided_harm": 721, + "selected_both_correct": 552, + "selected_both_wrong": 152 + }, + { + "budget": 0.11, + "threshold": 1.2173664569854736, + "operator": "ge", + "actual_fraction": 0.11284953395472704, + "metric": 0.20834032699131655, + "rescued": 471, + "harmed": 62, + "missed_rescue": 811, + "avoided_harm": 715, + "selected_both_correct": 659, + "selected_both_wrong": 164 + }, + { + "budget": 0.12, + "threshold": 1.1719112396240234, + "operator": "ge", + "actual_fraction": 0.12350199733688415, + "metric": 0.21121674052765757, + "rescued": 503, + "harmed": 67, + "missed_rescue": 779, + "avoided_harm": 710, + "selected_both_correct": 745, + "selected_both_wrong": 169 + }, + { + "budget": 0.13, + "threshold": 1.1186922788619995, + "operator": "ge", + "actual_fraction": 0.13315579227696406, + "metric": 0.21276215593410724, + "rescued": 531, + "harmed": 77, + "missed_rescue": 751, + "avoided_harm": 700, + "selected_both_correct": 818, + "selected_both_wrong": 174 + }, + { + "budget": 0.14, + "threshold": 1.0748379230499268, + "operator": "ge", + "actual_fraction": 0.14347536617842876, + "metric": 0.21511521078428622, + "rescued": 556, + "harmed": 79, + "missed_rescue": 726, + "avoided_harm": 698, + "selected_both_correct": 907, + "selected_both_wrong": 182 + }, + { + "budget": 0.15, + "threshold": 1.0427184104919434, + "operator": "ge", + "actual_fraction": 0.15246338215712382, + "metric": 0.21738179676765032, + "rescued": 577, + "harmed": 81, + "missed_rescue": 705, + "avoided_harm": 696, + "selected_both_correct": 988, + "selected_both_wrong": 186 + }, + { + "budget": 0.16, + "threshold": 1.0111123323440552, + "operator": "ge", + "actual_fraction": 0.1604527296937417, + "metric": 0.22260454951009195, + "rescued": 597, + "harmed": 83, + "missed_rescue": 685, + "avoided_harm": 694, + "selected_both_correct": 1055, + "selected_both_wrong": 193 + }, + { + "budget": 0.17, + "threshold": 0.9669641852378845, + "operator": "ge", + "actual_fraction": 0.1780958721704394, + "metric": 0.22627631882842034, + "rescued": 641, + "harmed": 97, + "missed_rescue": 641, + "avoided_harm": 680, + "selected_both_correct": 1196, + "selected_both_wrong": 206 + }, + { + "budget": 0.18, + "threshold": 0.9490972757339478, + "operator": "ge", + "actual_fraction": 0.18442077230359522, + "metric": 0.2270967395967396, + "rescued": 657, + "harmed": 103, + "missed_rescue": 625, + "avoided_harm": 674, + "selected_both_correct": 1246, + "selected_both_wrong": 210 + }, + { + "budget": 0.19, + "threshold": 0.9165292382240295, + "operator": "ge", + "actual_fraction": 0.2000665778961385, + "metric": 0.23397114982462924, + "rescued": 697, + "harmed": 110, + "missed_rescue": 585, + "avoided_harm": 667, + "selected_both_correct": 1378, + "selected_both_wrong": 219 + }, + { + "budget": 0.2, + "threshold": 0.8978593349456787, + "operator": "ge", + "actual_fraction": 0.20439414114513982, + "metric": 0.23512276426997616, + "rescued": 707, + "harmed": 112, + "missed_rescue": 575, + "avoided_harm": 665, + "selected_both_correct": 1413, + "selected_both_wrong": 224 + }, + { + "budget": 0.21, + "threshold": 0.854847252368927, + "operator": "ge", + "actual_fraction": 0.21937416777629826, + "metric": 0.24202469545559713, + "rescued": 747, + "harmed": 119, + "missed_rescue": 535, + "avoided_harm": 658, + "selected_both_correct": 1539, + "selected_both_wrong": 231 + }, + { + "budget": 0.22, + "threshold": 0.8254434466362, + "operator": "ge", + "actual_fraction": 0.22769640479360853, + "metric": 0.24353835097564847, + "rescued": 766, + "harmed": 124, + "missed_rescue": 516, + "avoided_harm": 653, + "selected_both_correct": 1611, + "selected_both_wrong": 235 + }, + { + "budget": 0.23, + "threshold": 0.8046320080757141, + "operator": "ge", + "actual_fraction": 0.23601864181091878, + "metric": 0.2454657390209296, + "rescued": 787, + "harmed": 129, + "missed_rescue": 495, + "avoided_harm": 648, + "selected_both_correct": 1680, + "selected_both_wrong": 240 + }, + { + "budget": 0.24, + "threshold": 0.7761231660842896, + "operator": "ge", + "actual_fraction": 0.24533954727030627, + "metric": 0.2483414219175777, + "rescued": 812, + "harmed": 133, + "missed_rescue": 470, + "avoided_harm": 644, + "selected_both_correct": 1760, + "selected_both_wrong": 243 + }, + { + "budget": 0.25, + "threshold": 0.7452236413955688, + "operator": "ge", + "actual_fraction": 0.25432756324900135, + "metric": 0.2504846177578749, + "rescued": 831, + "harmed": 135, + "missed_rescue": 451, + "avoided_harm": 642, + "selected_both_correct": 1839, + "selected_both_wrong": 251 + }, + { + "budget": 0.26, + "threshold": 0.7167489528656006, + "operator": "ge", + "actual_fraction": 0.2636484687083888, + "metric": 0.252142216376585, + "rescued": 855, + "harmed": 144, + "missed_rescue": 427, + "avoided_harm": 633, + "selected_both_correct": 1914, + "selected_both_wrong": 255 + }, + { + "budget": 0.27, + "threshold": 0.6837625503540039, + "operator": "ge", + "actual_fraction": 0.275965379494008, + "metric": 0.25276746625574176, + "rescued": 880, + "harmed": 155, + "missed_rescue": 402, + "avoided_harm": 622, + "selected_both_correct": 2019, + "selected_both_wrong": 262 + }, + { + "budget": 0.28, + "threshold": 0.633089005947113, + "operator": "ge", + "actual_fraction": 0.29427430093209056, + "metric": 0.25702642690899347, + "rescued": 927, + "harmed": 172, + "missed_rescue": 355, + "avoided_harm": 605, + "selected_both_correct": 2167, + "selected_both_wrong": 270 + }, + { + "budget": 0.29, + "threshold": 0.5875619649887085, + "operator": "ge", + "actual_fraction": 0.31091877496671105, + "metric": 0.26183035419940204, + "rescued": 969, + "harmed": 193, + "missed_rescue": 313, + "avoided_harm": 584, + "selected_both_correct": 2296, + "selected_both_wrong": 278 + }, + { + "budget": 0.3, + "threshold": 0.5604347586631775, + "operator": "ge", + "actual_fraction": 0.322237017310253, + "metric": 0.2672887365132105, + "rescued": 995, + "harmed": 204, + "missed_rescue": 287, + "avoided_harm": 573, + "selected_both_correct": 2389, + "selected_both_wrong": 284 + }, + { + "budget": 0.31, + "threshold": 0.5320743918418884, + "operator": "ge", + "actual_fraction": 0.33388814913448733, + "metric": 0.27039536303602457, + "rescued": 1020, + "harmed": 216, + "missed_rescue": 262, + "avoided_harm": 561, + "selected_both_correct": 2480, + "selected_both_wrong": 296 + }, + { + "budget": 0.32, + "threshold": 0.49417412281036377, + "operator": "ge", + "actual_fraction": 0.34354194407456723, + "metric": 0.2713201570387918, + "rescued": 1038, + "harmed": 227, + "missed_rescue": 244, + "avoided_harm": 550, + "selected_both_correct": 2562, + "selected_both_wrong": 301 + }, + { + "budget": 0.33, + "threshold": 0.46743881702423096, + "operator": "ge", + "actual_fraction": 0.3492010652463382, + "metric": 0.2722481828097158, + "rescued": 1046, + "harmed": 230, + "missed_rescue": 236, + "avoided_harm": 547, + "selected_both_correct": 2610, + "selected_both_wrong": 310 + }, + { + "budget": 0.34, + "threshold": 0.4429023861885071, + "operator": "ge", + "actual_fraction": 0.3545272969374168, + "metric": 0.274417471554269, + "rescued": 1061, + "harmed": 239, + "missed_rescue": 221, + "avoided_harm": 538, + "selected_both_correct": 2648, + "selected_both_wrong": 312 + }, + { + "budget": 0.35, + "threshold": 0.40694165229797363, + "operator": "ge", + "actual_fraction": 0.3648468708388815, + "metric": 0.2782223498666452, + "rescued": 1081, + "harmed": 252, + "missed_rescue": 201, + "avoided_harm": 525, + "selected_both_correct": 2730, + "selected_both_wrong": 321 + }, + { + "budget": 0.36, + "threshold": 0.3611782193183899, + "operator": "ge", + "actual_fraction": 0.37217043941411454, + "metric": 0.2795152621397537, + "rescued": 1094, + "harmed": 257, + "missed_rescue": 188, + "avoided_harm": 520, + "selected_both_correct": 2793, + "selected_both_wrong": 328 + }, + { + "budget": 0.37, + "threshold": 0.3311936855316162, + "operator": "ge", + "actual_fraction": 0.37683089214380827, + "metric": 0.28090947933574756, + "rescued": 1104, + "harmed": 261, + "missed_rescue": 178, + "avoided_harm": 516, + "selected_both_correct": 2831, + "selected_both_wrong": 332 + }, + { + "budget": 0.38, + "threshold": 0.28899651765823364, + "operator": "ge", + "actual_fraction": 0.3854860186418109, + "metric": 0.28295009794869297, + "rescued": 1121, + "harmed": 269, + "missed_rescue": 161, + "avoided_harm": 508, + "selected_both_correct": 2904, + "selected_both_wrong": 338 + }, + { + "budget": 0.39, + "threshold": 0.19986218214035034, + "operator": "ge", + "actual_fraction": 0.39980026631158455, + "metric": 0.28644294895435674, + "rescued": 1145, + "harmed": 278, + "missed_rescue": 137, + "avoided_harm": 499, + "selected_both_correct": 3027, + "selected_both_wrong": 354 + }, + { + "budget": 0.4, + "threshold": 0.157886803150177, + "operator": "ge", + "actual_fraction": 0.4054593874833555, + "metric": 0.2862200047733393, + "rescued": 1160, + "harmed": 284, + "missed_rescue": 122, + "avoided_harm": 493, + "selected_both_correct": 3072, + "selected_both_wrong": 356 + }, + { + "budget": 0.41, + "threshold": 0.12043046951293945, + "operator": "ge", + "actual_fraction": 0.4104527296937417, + "metric": 0.28760386086589435, + "rescued": 1168, + "harmed": 287, + "missed_rescue": 114, + "avoided_harm": 490, + "selected_both_correct": 3115, + "selected_both_wrong": 362 + }, + { + "budget": 0.42, + "threshold": 0.06623327732086182, + "operator": "ge", + "actual_fraction": 0.4181091877496671, + "metric": 0.2930744270093899, + "rescued": 1181, + "harmed": 293, + "missed_rescue": 101, + "avoided_harm": 484, + "selected_both_correct": 3180, + "selected_both_wrong": 370 + }, + { + "budget": 0.43, + "threshold": -0.01827472448348999, + "operator": "ge", + "actual_fraction": 0.42809587217043943, + "metric": 0.2961685540536353, + "rescued": 1200, + "harmed": 302, + "missed_rescue": 82, + "avoided_harm": 475, + "selected_both_correct": 3262, + "selected_both_wrong": 380 + }, + { + "budget": 0.44, + "threshold": -0.14177680015563965, + "operator": "ge", + "actual_fraction": 0.4370838881491345, + "metric": 0.2977058723546182, + "rescued": 1215, + "harmed": 309, + "missed_rescue": 67, + "avoided_harm": 468, + "selected_both_correct": 3338, + "selected_both_wrong": 390 + }, + { + "budget": 0.45, + "threshold": -0.270923376083374, + "operator": "ge", + "actual_fraction": 0.44673768308921435, + "metric": 0.2999402430635043, + "rescued": 1230, + "harmed": 313, + "missed_rescue": 52, + "avoided_harm": 464, + "selected_both_correct": 3422, + "selected_both_wrong": 403 + }, + { + "budget": 0.46, + "threshold": -0.4213377833366394, + "operator": "ge", + "actual_fraction": 0.45206391478029295, + "metric": 0.2993046120720782, + "rescued": 1235, + "harmed": 316, + "missed_rescue": 47, + "avoided_harm": 461, + "selected_both_correct": 3475, + "selected_both_wrong": 406 + }, + { + "budget": 0.47, + "threshold": -0.6602996587753296, + "operator": "ge", + "actual_fraction": 0.45872170439414117, + "metric": 0.2998310238962732, + "rescued": 1243, + "harmed": 321, + "missed_rescue": 39, + "avoided_harm": 456, + "selected_both_correct": 3536, + "selected_both_wrong": 412 + }, + { + "budget": 0.48, + "threshold": -0.8612939715385437, + "operator": "ge", + "actual_fraction": 0.4673768308921438, + "metric": 0.30393211265289466, + "rescued": 1247, + "harmed": 326, + "missed_rescue": 35, + "avoided_harm": 451, + "selected_both_correct": 3623, + "selected_both_wrong": 420 + }, + { + "budget": 0.49, + "threshold": -0.9715782999992371, + "operator": "ge", + "actual_fraction": 0.47503328894806923, + "metric": 0.30350001249166053, + "rescued": 1248, + "harmed": 328, + "missed_rescue": 34, + "avoided_harm": 449, + "selected_both_correct": 3710, + "selected_both_wrong": 422 + }, + { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4876830892143808, + "metric": 0.29875800068449215, + "rescued": 1249, + "harmed": 337, + "missed_rescue": 33, + "avoided_harm": 440, + "selected_both_correct": 3843, + "selected_both_wrong": 431 + }, + { + "budget": 0.51, + "threshold": -1.6505446434020996, + "operator": "ge", + "actual_fraction": 0.5046604527296937, + "metric": 0.29989697064322696, + "rescued": 1255, + "harmed": 348, + "missed_rescue": 27, + "avoided_harm": 429, + "selected_both_correct": 4026, + "selected_both_wrong": 435 + }, + { + "budget": 0.52, + "threshold": -2.6140921115875244, + "operator": "ge", + "actual_fraction": 0.5206391478029294, + "metric": 0.297424793956044, + "rescued": 1255, + "harmed": 359, + "missed_rescue": 27, + "avoided_harm": 418, + "selected_both_correct": 4205, + "selected_both_wrong": 437 + }, + { + "budget": 0.53, + "threshold": -2.6907107830047607, + "operator": "ge", + "actual_fraction": 0.5249667110519307, + "metric": 0.297424793956044, + "rescued": 1255, + "harmed": 359, + "missed_rescue": 27, + "avoided_harm": 418, + "selected_both_correct": 4257, + "selected_both_wrong": 437 + }, + { + "budget": 0.54, + "threshold": -2.8344802856445312, + "operator": "ge", + "actual_fraction": 0.5436085219707057, + "metric": 0.2978194775414123, + "rescued": 1257, + "harmed": 377, + "missed_rescue": 25, + "avoided_harm": 400, + "selected_both_correct": 4461, + "selected_both_wrong": 437 + }, + { + "budget": 0.55, + "threshold": -2.905412197113037, + "operator": "ge", + "actual_fraction": 0.5529294274300932, + "metric": 0.29630990060928936, + "rescued": 1257, + "harmed": 385, + "missed_rescue": 25, + "avoided_harm": 392, + "selected_both_correct": 4565, + "selected_both_wrong": 437 + }, + { + "budget": 0.56, + "threshold": -2.9482638835906982, + "operator": "ge", + "actual_fraction": 0.5612516644474035, + "metric": 0.29578344205458756, + "rescued": 1257, + "harmed": 388, + "missed_rescue": 25, + "avoided_harm": 389, + "selected_both_correct": 4660, + "selected_both_wrong": 439 + }, + { + "budget": 0.57, + "threshold": -2.9957408905029297, + "operator": "ge", + "actual_fraction": 0.5692410119840213, + "metric": 0.2950949302773625, + "rescued": 1257, + "harmed": 396, + "missed_rescue": 25, + "avoided_harm": 381, + "selected_both_correct": 4748, + "selected_both_wrong": 439 + }, + { + "budget": 0.58, + "threshold": -3.1037726402282715, + "operator": "ge", + "actual_fraction": 0.5902130492676432, + "metric": 0.2947373376338526, + "rescued": 1260, + "harmed": 407, + "missed_rescue": 22, + "avoided_harm": 370, + "selected_both_correct": 4985, + "selected_both_wrong": 440 + }, + { + "budget": 0.59, + "threshold": -3.1757895946502686, + "operator": "ge", + "actual_fraction": 0.6021970705725699, + "metric": 0.29722091277674884, + "rescued": 1261, + "harmed": 417, + "missed_rescue": 21, + "avoided_harm": 360, + "selected_both_correct": 5115, + "selected_both_wrong": 443 + }, + { + "budget": 0.6, + "threshold": -3.253206729888916, + "operator": "ge", + "actual_fraction": 0.6208388814913449, + "metric": 0.29665021956289706, + "rescued": 1264, + "harmed": 430, + "missed_rescue": 18, + "avoided_harm": 347, + "selected_both_correct": 5323, + "selected_both_wrong": 443 + }, + { + "budget": 0.61, + "threshold": -3.2825026512145996, + "operator": "ge", + "actual_fraction": 0.6288282290279628, + "metric": 0.295988506182091, + "rescued": 1264, + "harmed": 435, + "missed_rescue": 18, + "avoided_harm": 342, + "selected_both_correct": 5413, + "selected_both_wrong": 444 + }, + { + "budget": 0.62, + "threshold": -3.322240114212036, + "operator": "ge", + "actual_fraction": 0.6421438082556591, + "metric": 0.2939669175161438, + "rescued": 1264, + "harmed": 452, + "missed_rescue": 18, + "avoided_harm": 325, + "selected_both_correct": 5556, + "selected_both_wrong": 444 + }, + { + "budget": 0.63, + "threshold": -3.3503003120422363, + "operator": "ge", + "actual_fraction": 0.6491344873501997, + "metric": 0.29277362668991463, + "rescued": 1265, + "harmed": 463, + "missed_rescue": 17, + "avoided_harm": 314, + "selected_both_correct": 5628, + "selected_both_wrong": 444 + }, + { + "budget": 0.64, + "threshold": -3.377089738845825, + "operator": "ge", + "actual_fraction": 0.6571238348868176, + "metric": 0.2930821542583563, + "rescued": 1266, + "harmed": 468, + "missed_rescue": 16, + "avoided_harm": 309, + "selected_both_correct": 5718, + "selected_both_wrong": 444 + }, + { + "budget": 0.65, + "threshold": -3.4055864810943604, + "operator": "ge", + "actual_fraction": 0.6677762982689747, + "metric": 0.2892305220657417, + "rescued": 1266, + "harmed": 488, + "missed_rescue": 16, + "avoided_harm": 289, + "selected_both_correct": 5826, + "selected_both_wrong": 444 + }, + { + "budget": 0.66, + "threshold": -3.430248498916626, + "operator": "ge", + "actual_fraction": 0.6744340878828229, + "metric": 0.2915915088317972, + "rescued": 1267, + "harmed": 492, + "missed_rescue": 15, + "avoided_harm": 285, + "selected_both_correct": 5901, + "selected_both_wrong": 444 + }, + { + "budget": 0.67, + "threshold": -3.4584414958953857, + "operator": "ge", + "actual_fraction": 0.6854194407456724, + "metric": 0.2936540392680743, + "rescued": 1269, + "harmed": 501, + "missed_rescue": 13, + "avoided_harm": 276, + "selected_both_correct": 6022, + "selected_both_wrong": 444 + }, + { + "budget": 0.68, + "threshold": -3.502922773361206, + "operator": "ge", + "actual_fraction": 0.7047270306258322, + "metric": 0.29632332439424064, + "rescued": 1272, + "harmed": 523, + "missed_rescue": 10, + "avoided_harm": 254, + "selected_both_correct": 6229, + "selected_both_wrong": 444 + }, + { + "budget": 0.69, + "threshold": -3.532865285873413, + "operator": "ge", + "actual_fraction": 0.7157123834886817, + "metric": 0.2937246709263134, + "rescued": 1273, + "harmed": 540, + "missed_rescue": 9, + "avoided_harm": 237, + "selected_both_correct": 6342, + "selected_both_wrong": 445 + }, + { + "budget": 0.7, + "threshold": -3.5550942420959473, + "operator": "ge", + "actual_fraction": 0.7253661784287616, + "metric": 0.2952254059747369, + "rescued": 1274, + "harmed": 550, + "missed_rescue": 8, + "avoided_harm": 227, + "selected_both_correct": 6446, + "selected_both_wrong": 446 + }, + { + "budget": 0.71, + "threshold": -3.566344976425171, + "operator": "ge", + "actual_fraction": 0.7283621837549934, + "metric": 0.2945683782087344, + "rescued": 1274, + "harmed": 553, + "missed_rescue": 8, + "avoided_harm": 224, + "selected_both_correct": 6478, + "selected_both_wrong": 447 + }, + { + "budget": 0.72, + "threshold": -3.6056067943573, + "operator": "ge", + "actual_fraction": 0.741011984021305, + "metric": 0.29252963134197957, + "rescued": 1275, + "harmed": 568, + "missed_rescue": 7, + "avoided_harm": 209, + "selected_both_correct": 6614, + "selected_both_wrong": 447 + }, + { + "budget": 0.73, + "threshold": -3.628530979156494, + "operator": "ge", + "actual_fraction": 0.7526631158455392, + "metric": 0.2900395106945919, + "rescued": 1276, + "harmed": 588, + "missed_rescue": 6, + "avoided_harm": 189, + "selected_both_correct": 6733, + "selected_both_wrong": 447 + }, + { + "budget": 0.74, + "threshold": -3.6580705642700195, + "operator": "ge", + "actual_fraction": 0.7616511318242344, + "metric": 0.2887744568626224, + "rescued": 1276, + "harmed": 597, + "missed_rescue": 6, + "avoided_harm": 180, + "selected_both_correct": 6832, + "selected_both_wrong": 447 + }, + { + "budget": 0.75, + "threshold": -3.6900978088378906, + "operator": "ge", + "actual_fraction": 0.7709720372836218, + "metric": 0.28730364578560735, + "rescued": 1276, + "harmed": 606, + "missed_rescue": 6, + "avoided_harm": 171, + "selected_both_correct": 6935, + "selected_both_wrong": 447 + }, + { + "budget": 0.76, + "threshold": -3.7050747871398926, + "operator": "ge", + "actual_fraction": 0.7749667110519307, + "metric": 0.2858070539975931, + "rescued": 1276, + "harmed": 612, + "missed_rescue": 6, + "avoided_harm": 165, + "selected_both_correct": 6977, + "selected_both_wrong": 447 + }, + { + "budget": 0.77, + "threshold": -3.720116138458252, + "operator": "ge", + "actual_fraction": 0.7792942743009321, + "metric": 0.28488735368132817, + "rescued": 1276, + "harmed": 617, + "missed_rescue": 6, + "avoided_harm": 160, + "selected_both_correct": 7023, + "selected_both_wrong": 448 + }, + { + "budget": 0.78, + "threshold": -3.7399919033050537, + "operator": "ge", + "actual_fraction": 0.7846205059920106, + "metric": 0.2830658409372236, + "rescued": 1276, + "harmed": 627, + "missed_rescue": 6, + "avoided_harm": 150, + "selected_both_correct": 7077, + "selected_both_wrong": 448 + }, + { + "budget": 0.79, + "threshold": -3.799569606781006, + "operator": "ge", + "actual_fraction": 0.7972703062583223, + "metric": 0.2809995077380739, + "rescued": 1277, + "harmed": 643, + "missed_rescue": 5, + "avoided_harm": 134, + "selected_both_correct": 7212, + "selected_both_wrong": 448 + }, + { + "budget": 0.8, + "threshold": -3.8980000019073486, + "operator": "ge", + "actual_fraction": 0.8129161118508655, + "metric": 0.28013559922601305, + "rescued": 1278, + "harmed": 667, + "missed_rescue": 4, + "avoided_harm": 110, + "selected_both_correct": 7375, + "selected_both_wrong": 448 + }, + { + "budget": 0.81, + "threshold": -4.0607757568359375, + "operator": "ge", + "actual_fraction": 0.8199067909454061, + "metric": 0.27814841049220523, + "rescued": 1278, + "harmed": 681, + "missed_rescue": 4, + "avoided_harm": 96, + "selected_both_correct": 7445, + "selected_both_wrong": 448 + }, + { + "budget": 0.82, + "threshold": -8.964031219482422, + "operator": "ge", + "actual_fraction": 0.831225033288948, + "metric": 0.2772348763322643, + "rescued": 1278, + "harmed": 686, + "missed_rescue": 4, + "avoided_harm": 91, + "selected_both_correct": 7576, + "selected_both_wrong": 448 + }, + { + "budget": 0.83, + "threshold": -9.02132511138916, + "operator": "ge", + "actual_fraction": 0.84254327563249, + "metric": 0.27700595116508253, + "rescued": 1279, + "harmed": 689, + "missed_rescue": 3, + "avoided_harm": 88, + "selected_both_correct": 7708, + "selected_both_wrong": 448 + }, + { + "budget": 0.84, + "threshold": -9.09000015258789, + "operator": "ge", + "actual_fraction": 0.857190412782956, + "metric": 0.2763733409861609, + "rescued": 1279, + "harmed": 692, + "missed_rescue": 3, + "avoided_harm": 85, + "selected_both_correct": 7881, + "selected_both_wrong": 448 + }, + { + "budget": 0.85, + "threshold": -9.12938404083252, + "operator": "ge", + "actual_fraction": 0.8618508655126498, + "metric": 0.2761345157638265, + "rescued": 1279, + "harmed": 694, + "missed_rescue": 3, + "avoided_harm": 83, + "selected_both_correct": 7935, + "selected_both_wrong": 448 + }, + { + "budget": 0.86, + "threshold": -9.17154312133789, + "operator": "ge", + "actual_fraction": 0.87017310252996, + "metric": 0.2761345157638265, + "rescued": 1279, + "harmed": 694, + "missed_rescue": 3, + "avoided_harm": 83, + "selected_both_correct": 8035, + "selected_both_wrong": 448 + }, + { + "budget": 0.87, + "threshold": -9.273024559020996, + "operator": "ge", + "actual_fraction": 0.8848202396804261, + "metric": 0.27546832609774896, + "rescued": 1279, + "harmed": 699, + "missed_rescue": 3, + "avoided_harm": 78, + "selected_both_correct": 8206, + "selected_both_wrong": 448 + }, + { + "budget": 0.88, + "threshold": -9.444879531860352, + "operator": "ge", + "actual_fraction": 0.8984687083888149, + "metric": 0.27423981143437465, + "rescued": 1279, + "harmed": 709, + "missed_rescue": 3, + "avoided_harm": 68, + "selected_both_correct": 8360, + "selected_both_wrong": 448 + }, + { + "budget": 0.89, + "threshold": -9.566991806030273, + "operator": "ge", + "actual_fraction": 0.9087882822902796, + "metric": 0.2736575311942959, + "rescued": 1279, + "harmed": 713, + "missed_rescue": 3, + "avoided_harm": 64, + "selected_both_correct": 8480, + "selected_both_wrong": 448 + }, + { + "budget": 0.9, + "threshold": -9.675031661987305, + "operator": "ge", + "actual_fraction": 0.9161118508655126, + "metric": 0.272975172975173, + "rescued": 1279, + "harmed": 717, + "missed_rescue": 3, + "avoided_harm": 60, + "selected_both_correct": 8563, + "selected_both_wrong": 449 + }, + { + "budget": 0.91, + "threshold": -9.739017486572266, + "operator": "ge", + "actual_fraction": 0.9217709720372836, + "metric": 0.2727190231806067, + "rescued": 1279, + "harmed": 719, + "missed_rescue": 3, + "avoided_harm": 58, + "selected_both_correct": 8629, + "selected_both_wrong": 449 + }, + { + "budget": 0.92, + "threshold": -9.787779808044434, + "operator": "ge", + "actual_fraction": 0.9264314247669774, + "metric": 0.27264729817760425, + "rescued": 1279, + "harmed": 720, + "missed_rescue": 3, + "avoided_harm": 57, + "selected_both_correct": 8684, + "selected_both_wrong": 449 + }, + { + "budget": 0.93, + "threshold": -9.832305908203125, + "operator": "ge", + "actual_fraction": 0.933422103861518, + "metric": 0.2741441928343702, + "rescued": 1280, + "harmed": 724, + "missed_rescue": 2, + "avoided_harm": 53, + "selected_both_correct": 8763, + "selected_both_wrong": 449 + }, + { + "budget": 0.94, + "threshold": -9.88642692565918, + "operator": "ge", + "actual_fraction": 0.9417443408788282, + "metric": 0.2733716828250215, + "rescued": 1280, + "harmed": 728, + "missed_rescue": 2, + "avoided_harm": 49, + "selected_both_correct": 8859, + "selected_both_wrong": 449 + }, + { + "budget": 0.95, + "threshold": -9.976344108581543, + "operator": "ge", + "actual_fraction": 0.9530625832223701, + "metric": 0.2727312493845365, + "rescued": 1280, + "harmed": 737, + "missed_rescue": 2, + "avoided_harm": 40, + "selected_both_correct": 8986, + "selected_both_wrong": 449 + }, + { + "budget": 0.96, + "threshold": -10.03441047668457, + "operator": "ge", + "actual_fraction": 0.9613848202396804, + "metric": 0.27223682114986464, + "rescued": 1280, + "harmed": 740, + "missed_rescue": 2, + "avoided_harm": 37, + "selected_both_correct": 9083, + "selected_both_wrong": 449 + }, + { + "budget": 0.97, + "threshold": -10.085777282714844, + "operator": "ge", + "actual_fraction": 0.9673768308921438, + "metric": 0.27453093162048386, + "rescued": 1282, + "harmed": 744, + "missed_rescue": 0, + "avoided_harm": 33, + "selected_both_correct": 9149, + "selected_both_wrong": 449 + }, + { + "budget": 0.98, + "threshold": -10.1488676071167, + "operator": "ge", + "actual_fraction": 0.9760319573901465, + "metric": 0.2737331615154065, + "rescued": 1282, + "harmed": 750, + "missed_rescue": 0, + "avoided_harm": 27, + "selected_both_correct": 9247, + "selected_both_wrong": 449 + }, + { + "budget": 0.99, + "threshold": -10.265552520751953, + "operator": "ge", + "actual_fraction": 0.9890146471371505, + "metric": 0.2725383412883413, + "rescued": 1282, + "harmed": 758, + "missed_rescue": 0, + "avoided_harm": 19, + "selected_both_correct": 9395, + "selected_both_wrong": 449 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.27001009067886245, + "rescued": 1282, + "harmed": 777, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9508, + "selected_both_wrong": 449 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 2189, + "rescuable": 307, + "harmful": 220, + "both_wrong": 88, + "unit": "event", + "total": 2804 + }, + "test": { + "both_correct": 9508, + "rescuable": 1282, + "harmful": 777, + "both_wrong": 449, + "unit": "event", + "total": 12016 + } + }, + "small": { + "validation": 0.18254072168965788, + "test": 0.18129624718318377 + }, + "large": { + "validation": 0.25448626222520154, + "test": 0.27001009067886245 + }, + "learned_minus_confidence_test": 0.03900900020828163, + "learned_minus_small_test": 0.11746175350130839, + "learned_minus_large_test": 0.028747910005629707 + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "MiniCPM" + ], + "expert_display": "RT-DETR-X", + "run": "shared_12_rtdetr_x_fp32_MiniCPM", + "validation_rows": 701, + "test_rows": 3004, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detection_count", + "has_detections", + "max_score", + "mean_score", + "detection_count_valid", + "has_detections_valid", + "max_score_valid", + "mean_score_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/rtdetr_x_fp32/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 98, + "validation_area_0_25": 0.08994758708890332, + "validation_point": { + "budget": 0.98, + "threshold": -0.4221048653125763, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.20572745422879232, + "rescued": 770, + "harmed": 465, + "missed_rescue": 16, + "avoided_harm": 10, + "selected_both_correct": 1366, + "selected_both_wrong": 143 + }, + "test_point": { + "budget": 0.98, + "threshold": -0.4221048653125763, + "operator": "ge", + "actual_fraction": 0.9790279627163782, + "metric": 0.21609512872414097, + "rescued": 3244, + "harmed": 1926, + "missed_rescue": 70, + "avoided_harm": 19, + "selected_both_correct": 5928, + "selected_both_wrong": 666 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.07641657905799261, + "rescued": 0, + "harmed": 0, + "missed_rescue": 786, + "avoided_harm": 475, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.021043073385953903, + "operator": "ge", + "actual_fraction": 0.009985734664764621, + "metric": 0.07628160544643427, + "rescued": 0, + "harmed": 5, + "missed_rescue": 786, + "avoided_harm": 470, + "selected_both_correct": 23, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -0.022327054291963577, + "operator": "ge", + "actual_fraction": 0.019971469329529243, + "metric": 0.07620479856453766, + "rescued": 0, + "harmed": 10, + "missed_rescue": 786, + "avoided_harm": 465, + "selected_both_correct": 46, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -0.024909960106015205, + "operator": "ge", + "actual_fraction": 0.029957203994293864, + "metric": 0.076179559423678, + "rescued": 0, + "harmed": 16, + "missed_rescue": 786, + "avoided_harm": 459, + "selected_both_correct": 68, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": -0.027814723551273346, + "operator": "ge", + "actual_fraction": 0.039942938659058486, + "metric": 0.07608036783512472, + "rescued": 0, + "harmed": 21, + "missed_rescue": 786, + "avoided_harm": 454, + "selected_both_correct": 91, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": -0.028914429247379303, + "operator": "ge", + "actual_fraction": 0.04992867332382311, + "metric": 0.09455032380508069, + "rescued": 1, + "harmed": 25, + "missed_rescue": 785, + "avoided_harm": 450, + "selected_both_correct": 114, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": -0.030170045793056488, + "operator": "ge", + "actual_fraction": 0.05991440798858773, + "metric": 0.09386491892417406, + "rescued": 1, + "harmed": 27, + "missed_rescue": 785, + "avoided_harm": 448, + "selected_both_correct": 139, + "selected_both_wrong": 1 + }, + { + "budget": 0.07, + "threshold": -0.03123854100704193, + "operator": "ge", + "actual_fraction": 0.06990014265335236, + "metric": 0.09204233166673514, + "rescued": 1, + "harmed": 34, + "missed_rescue": 785, + "avoided_harm": 441, + "selected_both_correct": 160, + "selected_both_wrong": 1 + }, + { + "budget": 0.08, + "threshold": -0.03410686179995537, + "operator": "ge", + "actual_fraction": 0.07988587731811697, + "metric": 0.091007586947048, + "rescued": 1, + "harmed": 38, + "missed_rescue": 785, + "avoided_harm": 437, + "selected_both_correct": 184, + "selected_both_wrong": 1 + }, + { + "budget": 0.09, + "threshold": -0.03476816788315773, + "operator": "ge", + "actual_fraction": 0.0898716119828816, + "metric": 0.08934652523756015, + "rescued": 1, + "harmed": 43, + "missed_rescue": 785, + "avoided_harm": 432, + "selected_both_correct": 207, + "selected_both_wrong": 1 + }, + { + "budget": 0.1, + "threshold": -0.0365506075322628, + "operator": "ge", + "actual_fraction": 0.09985734664764621, + "metric": 0.08860787026161702, + "rescued": 1, + "harmed": 47, + "missed_rescue": 785, + "avoided_harm": 428, + "selected_both_correct": 231, + "selected_both_wrong": 1 + }, + { + "budget": 0.11, + "threshold": -0.03818900138139725, + "operator": "ge", + "actual_fraction": 0.10984308131241084, + "metric": 0.0875161669606114, + "rescued": 1, + "harmed": 54, + "missed_rescue": 785, + "avoided_harm": 421, + "selected_both_correct": 252, + "selected_both_wrong": 1 + }, + { + "budget": 0.12, + "threshold": -0.039673544466495514, + "operator": "ge", + "actual_fraction": 0.11982881597717546, + "metric": 0.0869090348188258, + "rescued": 1, + "harmed": 63, + "missed_rescue": 785, + "avoided_harm": 412, + "selected_both_correct": 271, + "selected_both_wrong": 1 + }, + { + "budget": 0.13, + "threshold": -0.04085410386323929, + "operator": "ge", + "actual_fraction": 0.12981455064194009, + "metric": 0.08635025131899243, + "rescued": 1, + "harmed": 68, + "missed_rescue": 785, + "avoided_harm": 407, + "selected_both_correct": 294, + "selected_both_wrong": 1 + }, + { + "budget": 0.14, + "threshold": -0.04204101860523224, + "operator": "ge", + "actual_fraction": 0.1398002853067047, + "metric": 0.0857792865956536, + "rescued": 1, + "harmed": 75, + "missed_rescue": 785, + "avoided_harm": 400, + "selected_both_correct": 315, + "selected_both_wrong": 1 + }, + { + "budget": 0.15, + "threshold": -0.04278091713786125, + "operator": "ge", + "actual_fraction": 0.14978601997146934, + "metric": 0.09418918796431942, + "rescued": 2, + "harmed": 81, + "missed_rescue": 784, + "avoided_harm": 394, + "selected_both_correct": 335, + "selected_both_wrong": 2 + }, + { + "budget": 0.16, + "threshold": -0.04322759434580803, + "operator": "ge", + "actual_fraction": 0.15977175463623394, + "metric": 0.09315792624073146, + "rescued": 2, + "harmed": 85, + "missed_rescue": 784, + "avoided_harm": 390, + "selected_both_correct": 359, + "selected_both_wrong": 2 + }, + { + "budget": 0.17, + "threshold": -0.04394488036632538, + "operator": "ge", + "actual_fraction": 0.16975748930099857, + "metric": 0.09222958631972636, + "rescued": 2, + "harmed": 89, + "missed_rescue": 784, + "avoided_harm": 386, + "selected_both_correct": 383, + "selected_both_wrong": 2 + }, + { + "budget": 0.18, + "threshold": -0.04530126228928566, + "operator": "ge", + "actual_fraction": 0.1797432239657632, + "metric": 0.0917587567603033, + "rescued": 2, + "harmed": 95, + "missed_rescue": 784, + "avoided_harm": 380, + "selected_both_correct": 405, + "selected_both_wrong": 2 + }, + { + "budget": 0.19, + "threshold": -0.04661082476377487, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.09090632738556746, + "rescued": 2, + "harmed": 101, + "missed_rescue": 784, + "avoided_harm": 374, + "selected_both_correct": 427, + "selected_both_wrong": 2 + }, + { + "budget": 0.2, + "threshold": -0.04730507358908653, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.09027150780105647, + "rescued": 2, + "harmed": 106, + "missed_rescue": 784, + "avoided_harm": 369, + "selected_both_correct": 450, + "selected_both_wrong": 2 + }, + { + "budget": 0.21, + "threshold": -0.048246145248413086, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.08904603886890428, + "rescued": 2, + "harmed": 113, + "missed_rescue": 784, + "avoided_harm": 362, + "selected_both_correct": 471, + "selected_both_wrong": 2 + }, + { + "budget": 0.22, + "threshold": -0.04900278151035309, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.08819855956614042, + "rescued": 2, + "harmed": 121, + "missed_rescue": 784, + "avoided_harm": 354, + "selected_both_correct": 491, + "selected_both_wrong": 2 + }, + { + "budget": 0.23, + "threshold": -0.050042688846588135, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.08769891005218598, + "rescued": 2, + "harmed": 126, + "missed_rescue": 784, + "avoided_harm": 349, + "selected_both_correct": 513, + "selected_both_wrong": 3 + }, + { + "budget": 0.24, + "threshold": -0.050901688635349274, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.08812163994607147, + "rescued": 8, + "harmed": 130, + "missed_rescue": 778, + "avoided_harm": 345, + "selected_both_correct": 530, + "selected_both_wrong": 4 + }, + { + "budget": 0.25, + "threshold": -0.05161922052502632, + "operator": "ge", + "actual_fraction": 0.24964336661911554, + "metric": 0.08782151216689346, + "rescued": 14, + "harmed": 135, + "missed_rescue": 772, + "avoided_harm": 340, + "selected_both_correct": 546, + "selected_both_wrong": 5 + }, + { + "budget": 0.26, + "threshold": -0.05245445668697357, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.08802770491567173, + "rescued": 21, + "harmed": 139, + "missed_rescue": 765, + "avoided_harm": 336, + "selected_both_correct": 563, + "selected_both_wrong": 5 + }, + { + "budget": 0.27, + "threshold": -0.053079016506671906, + "operator": "ge", + "actual_fraction": 0.26961483594864477, + "metric": 0.08800890547752473, + "rescued": 28, + "harmed": 144, + "missed_rescue": 758, + "avoided_harm": 331, + "selected_both_correct": 578, + "selected_both_wrong": 6 + }, + { + "budget": 0.28, + "threshold": -0.053604189306497574, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.08849150328196631, + "rescued": 36, + "harmed": 148, + "missed_rescue": 750, + "avoided_harm": 327, + "selected_both_correct": 591, + "selected_both_wrong": 9 + }, + { + "budget": 0.29, + "threshold": -0.05505518987774849, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.0887205564920429, + "rescued": 46, + "harmed": 153, + "missed_rescue": 740, + "avoided_harm": 322, + "selected_both_correct": 604, + "selected_both_wrong": 9 + }, + { + "budget": 0.3, + "threshold": -0.05605518817901611, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.08901323629975644, + "rescued": 53, + "harmed": 157, + "missed_rescue": 733, + "avoided_harm": 318, + "selected_both_correct": 621, + "selected_both_wrong": 9 + }, + { + "budget": 0.31, + "threshold": -0.056986160576343536, + "operator": "ge", + "actual_fraction": 0.3095577746077033, + "metric": 0.08941457984183532, + "rescued": 61, + "harmed": 162, + "missed_rescue": 725, + "avoided_harm": 313, + "selected_both_correct": 635, + "selected_both_wrong": 10 + }, + { + "budget": 0.32, + "threshold": -0.0582931749522686, + "operator": "ge", + "actual_fraction": 0.3195435092724679, + "metric": 0.08986547999267158, + "rescued": 69, + "harmed": 167, + "missed_rescue": 717, + "avoided_harm": 308, + "selected_both_correct": 650, + "selected_both_wrong": 10 + }, + { + "budget": 0.33, + "threshold": -0.0596650205552578, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.0900623716189768, + "rescued": 76, + "harmed": 173, + "missed_rescue": 710, + "avoided_harm": 302, + "selected_both_correct": 664, + "selected_both_wrong": 11 + }, + { + "budget": 0.34, + "threshold": -0.060549296438694, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.09018872466182963, + "rescued": 87, + "harmed": 180, + "missed_rescue": 699, + "avoided_harm": 295, + "selected_both_correct": 673, + "selected_both_wrong": 12 + }, + { + "budget": 0.35, + "threshold": -0.06178095191717148, + "operator": "ge", + "actual_fraction": 0.34950071326676174, + "metric": 0.09513465140887797, + "rescued": 96, + "harmed": 184, + "missed_rescue": 690, + "avoided_harm": 291, + "selected_both_correct": 686, + "selected_both_wrong": 14 + }, + { + "budget": 0.36, + "threshold": -0.06328777223825455, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.09560176179757868, + "rescued": 108, + "harmed": 190, + "missed_rescue": 678, + "avoided_harm": 285, + "selected_both_correct": 696, + "selected_both_wrong": 14 + }, + { + "budget": 0.37, + "threshold": -0.0647548958659172, + "operator": "ge", + "actual_fraction": 0.369472182596291, + "metric": 0.09606601222778545, + "rescued": 118, + "harmed": 196, + "missed_rescue": 668, + "avoided_harm": 279, + "selected_both_correct": 707, + "selected_both_wrong": 15 + }, + { + "budget": 0.38, + "threshold": -0.06637481600046158, + "operator": "ge", + "actual_fraction": 0.37945791726105566, + "metric": 0.09683684024140088, + "rescued": 126, + "harmed": 199, + "missed_rescue": 660, + "avoided_harm": 276, + "selected_both_correct": 722, + "selected_both_wrong": 17 + }, + { + "budget": 0.39, + "threshold": -0.06774259358644485, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.09748138328334532, + "rescued": 135, + "harmed": 201, + "missed_rescue": 651, + "avoided_harm": 274, + "selected_both_correct": 736, + "selected_both_wrong": 20 + }, + { + "budget": 0.4, + "threshold": -0.06843380630016327, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.09725002075008596, + "rescued": 145, + "harmed": 205, + "missed_rescue": 641, + "avoided_harm": 270, + "selected_both_correct": 750, + "selected_both_wrong": 20 + }, + { + "budget": 0.41, + "threshold": -0.0694691464304924, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.09794654894022267, + "rescued": 155, + "harmed": 210, + "missed_rescue": 631, + "avoided_harm": 265, + "selected_both_correct": 763, + "selected_both_wrong": 20 + }, + { + "budget": 0.42, + "threshold": -0.07112707942724228, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.09874354292060185, + "rescued": 165, + "harmed": 216, + "missed_rescue": 621, + "avoided_harm": 259, + "selected_both_correct": 773, + "selected_both_wrong": 22 + }, + { + "budget": 0.43, + "threshold": -0.07192998379468918, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.09944895007868021, + "rescued": 174, + "harmed": 222, + "missed_rescue": 612, + "avoided_harm": 253, + "selected_both_correct": 785, + "selected_both_wrong": 23 + }, + { + "budget": 0.44, + "threshold": -0.07266208529472351, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.10024323502584373, + "rescued": 184, + "harmed": 226, + "missed_rescue": 602, + "avoided_harm": 249, + "selected_both_correct": 798, + "selected_both_wrong": 24 + }, + { + "budget": 0.45, + "threshold": -0.0744253620505333, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.10107630515550795, + "rescued": 194, + "harmed": 230, + "missed_rescue": 592, + "avoided_harm": 245, + "selected_both_correct": 810, + "selected_both_wrong": 26 + }, + { + "budget": 0.46, + "threshold": -0.07532940804958344, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.10181572307751467, + "rescued": 203, + "harmed": 235, + "missed_rescue": 583, + "avoided_harm": 240, + "selected_both_correct": 823, + "selected_both_wrong": 27 + }, + { + "budget": 0.47, + "threshold": -0.0770750343799591, + "operator": "ge", + "actual_fraction": 0.4693295292439372, + "metric": 0.10291838837461804, + "rescued": 218, + "harmed": 240, + "missed_rescue": 568, + "avoided_harm": 235, + "selected_both_correct": 830, + "selected_both_wrong": 28 + }, + { + "budget": 0.48, + "threshold": -0.07818037271499634, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.10398791390534362, + "rescued": 228, + "harmed": 245, + "missed_rescue": 558, + "avoided_harm": 230, + "selected_both_correct": 843, + "selected_both_wrong": 28 + }, + { + "budget": 0.49, + "threshold": -0.07998209446668625, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.1035500155095013, + "rescued": 235, + "harmed": 252, + "missed_rescue": 551, + "avoided_harm": 223, + "selected_both_correct": 857, + "selected_both_wrong": 28 + }, + { + "budget": 0.5, + "threshold": -0.08124949783086777, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.10310317656730472, + "rescued": 242, + "harmed": 256, + "missed_rescue": 544, + "avoided_harm": 219, + "selected_both_correct": 871, + "selected_both_wrong": 31 + }, + { + "budget": 0.51, + "threshold": -0.08398465067148209, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.10437739691349379, + "rescued": 254, + "harmed": 261, + "missed_rescue": 532, + "avoided_harm": 214, + "selected_both_correct": 881, + "selected_both_wrong": 32 + }, + { + "budget": 0.52, + "threshold": -0.08537434041500092, + "operator": "ge", + "actual_fraction": 0.5192582025677603, + "metric": 0.10682502852650196, + "rescued": 262, + "harmed": 268, + "missed_rescue": 524, + "avoided_harm": 207, + "selected_both_correct": 890, + "selected_both_wrong": 36 + }, + { + "budget": 0.53, + "threshold": -0.08664409816265106, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.10806066671218575, + "rescued": 273, + "harmed": 275, + "missed_rescue": 513, + "avoided_harm": 200, + "selected_both_correct": 899, + "selected_both_wrong": 37 + }, + { + "budget": 0.54, + "threshold": -0.08777308464050293, + "operator": "ge", + "actual_fraction": 0.5392296718972895, + "metric": 0.10917273744598222, + "rescued": 284, + "harmed": 278, + "missed_rescue": 502, + "avoided_harm": 197, + "selected_both_correct": 911, + "selected_both_wrong": 39 + }, + { + "budget": 0.55, + "threshold": -0.08980843424797058, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.11036769675516778, + "rescued": 296, + "harmed": 284, + "missed_rescue": 490, + "avoided_harm": 191, + "selected_both_correct": 920, + "selected_both_wrong": 40 + }, + { + "budget": 0.56, + "threshold": -0.0906188115477562, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.11423875293101385, + "rescued": 307, + "harmed": 287, + "missed_rescue": 479, + "avoided_harm": 188, + "selected_both_correct": 931, + "selected_both_wrong": 43 + }, + { + "budget": 0.57, + "threshold": -0.09160211682319641, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.11535691235491538, + "rescued": 317, + "harmed": 294, + "missed_rescue": 469, + "avoided_harm": 181, + "selected_both_correct": 940, + "selected_both_wrong": 45 + }, + { + "budget": 0.58, + "threshold": -0.09283093363046646, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.11669725808975426, + "rescued": 328, + "harmed": 298, + "missed_rescue": 458, + "avoided_harm": 177, + "selected_both_correct": 952, + "selected_both_wrong": 46 + }, + { + "budget": 0.59, + "threshold": -0.0941123366355896, + "operator": "ge", + "actual_fraction": 0.5891583452211127, + "metric": 0.1177679103161983, + "rescued": 338, + "harmed": 303, + "missed_rescue": 448, + "avoided_harm": 172, + "selected_both_correct": 964, + "selected_both_wrong": 47 + }, + { + "budget": 0.6, + "threshold": -0.09614869952201843, + "operator": "ge", + "actual_fraction": 0.5991440798858774, + "metric": 0.11921675545489711, + "rescued": 351, + "harmed": 309, + "missed_rescue": 435, + "avoided_harm": 166, + "selected_both_correct": 972, + "selected_both_wrong": 48 + }, + { + "budget": 0.61, + "threshold": -0.09772664308547974, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.12100079401350544, + "rescued": 364, + "harmed": 313, + "missed_rescue": 422, + "avoided_harm": 162, + "selected_both_correct": 981, + "selected_both_wrong": 50 + }, + { + "budget": 0.62, + "threshold": -0.09911532700061798, + "operator": "ge", + "actual_fraction": 0.6191155492154066, + "metric": 0.12498749148623647, + "rescued": 375, + "harmed": 315, + "missed_rescue": 411, + "avoided_harm": 160, + "selected_both_correct": 993, + "selected_both_wrong": 53 + }, + { + "budget": 0.63, + "threshold": -0.10007988661527634, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.12628905170512741, + "rescued": 387, + "harmed": 319, + "missed_rescue": 399, + "avoided_harm": 156, + "selected_both_correct": 1004, + "selected_both_wrong": 54 + }, + { + "budget": 0.64, + "threshold": -0.10177135467529297, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.12942700411115388, + "rescued": 403, + "harmed": 324, + "missed_rescue": 383, + "avoided_harm": 151, + "selected_both_correct": 1010, + "selected_both_wrong": 55 + }, + { + "budget": 0.65, + "threshold": -0.10395076125860214, + "operator": "ge", + "actual_fraction": 0.6490727532097005, + "metric": 0.13068664709512867, + "rescued": 410, + "harmed": 328, + "missed_rescue": 376, + "avoided_harm": 147, + "selected_both_correct": 1024, + "selected_both_wrong": 58 + }, + { + "budget": 0.66, + "threshold": -0.10524322092533112, + "operator": "ge", + "actual_fraction": 0.6590584878744651, + "metric": 0.1324582559366836, + "rescued": 422, + "harmed": 332, + "missed_rescue": 364, + "avoided_harm": 143, + "selected_both_correct": 1033, + "selected_both_wrong": 61 + }, + { + "budget": 0.67, + "threshold": -0.10689426958560944, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.13402207594539173, + "rescued": 435, + "harmed": 338, + "missed_rescue": 351, + "avoided_harm": 137, + "selected_both_correct": 1041, + "selected_both_wrong": 62 + }, + { + "budget": 0.68, + "threshold": -0.11014843732118607, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.13570216904654514, + "rescued": 447, + "harmed": 341, + "missed_rescue": 339, + "avoided_harm": 134, + "selected_both_correct": 1052, + "selected_both_wrong": 64 + }, + { + "budget": 0.69, + "threshold": -0.11117545515298843, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.13971639727625254, + "rescued": 458, + "harmed": 343, + "missed_rescue": 328, + "avoided_harm": 132, + "selected_both_correct": 1062, + "selected_both_wrong": 69 + }, + { + "budget": 0.7, + "threshold": -0.11230025440454483, + "operator": "ge", + "actual_fraction": 0.6990014265335235, + "metric": 0.14039490967751408, + "rescued": 469, + "harmed": 346, + "missed_rescue": 317, + "avoided_harm": 129, + "selected_both_correct": 1074, + "selected_both_wrong": 71 + }, + { + "budget": 0.71, + "threshold": -0.11423606425523758, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.1389440908530641, + "rescued": 476, + "harmed": 351, + "missed_rescue": 310, + "avoided_harm": 124, + "selected_both_correct": 1084, + "selected_both_wrong": 77 + }, + { + "budget": 0.72, + "threshold": -0.11647249013185501, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.1412819215041655, + "rescued": 492, + "harmed": 358, + "missed_rescue": 294, + "avoided_harm": 117, + "selected_both_correct": 1089, + "selected_both_wrong": 77 + }, + { + "budget": 0.73, + "threshold": -0.11962417513132095, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.14288929438332787, + "rescued": 503, + "harmed": 361, + "missed_rescue": 283, + "avoided_harm": 114, + "selected_both_correct": 1099, + "selected_both_wrong": 81 + }, + { + "budget": 0.74, + "threshold": -0.12126146256923676, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.14459529608777652, + "rescued": 514, + "harmed": 363, + "missed_rescue": 272, + "avoided_harm": 112, + "selected_both_correct": 1111, + "selected_both_wrong": 84 + }, + { + "budget": 0.75, + "threshold": -0.1230982318520546, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.14606069842667782, + "rescued": 525, + "harmed": 367, + "missed_rescue": 261, + "avoided_harm": 108, + "selected_both_correct": 1121, + "selected_both_wrong": 87 + }, + { + "budget": 0.76, + "threshold": -0.12517565488815308, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.14665677647800887, + "rescued": 537, + "harmed": 373, + "missed_rescue": 249, + "avoided_harm": 102, + "selected_both_correct": 1130, + "selected_both_wrong": 88 + }, + { + "budget": 0.77, + "threshold": -0.12628257274627686, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.1471300269894615, + "rescued": 546, + "harmed": 381, + "missed_rescue": 240, + "avoided_harm": 94, + "selected_both_correct": 1139, + "selected_both_wrong": 90 + }, + { + "budget": 0.78, + "threshold": -0.12975578010082245, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.1503016824168161, + "rescued": 558, + "harmed": 384, + "missed_rescue": 228, + "avoided_harm": 91, + "selected_both_correct": 1151, + "selected_both_wrong": 91 + }, + { + "budget": 0.79, + "threshold": -0.13229140639305115, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.15155183161566646, + "rescued": 566, + "harmed": 386, + "missed_rescue": 220, + "avoided_harm": 89, + "selected_both_correct": 1163, + "selected_both_wrong": 97 + }, + { + "budget": 0.8, + "threshold": -0.1373685747385025, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.15468771641968415, + "rescued": 579, + "harmed": 392, + "missed_rescue": 207, + "avoided_harm": 83, + "selected_both_correct": 1170, + "selected_both_wrong": 99 + }, + { + "budget": 0.81, + "threshold": -0.1407174915075302, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.16068290931933413, + "rescued": 597, + "harmed": 396, + "missed_rescue": 189, + "avoided_harm": 79, + "selected_both_correct": 1175, + "selected_both_wrong": 100 + }, + { + "budget": 0.82, + "threshold": -0.1433146893978119, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.16451572912479911, + "rescued": 612, + "harmed": 400, + "missed_rescue": 174, + "avoided_harm": 75, + "selected_both_correct": 1183, + "selected_both_wrong": 101 + }, + { + "budget": 0.83, + "threshold": -0.1457100659608841, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.16497266103579622, + "rescued": 619, + "harmed": 406, + "missed_rescue": 167, + "avoided_harm": 69, + "selected_both_correct": 1196, + "selected_both_wrong": 103 + }, + { + "budget": 0.84, + "threshold": -0.148710235953331, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.1662001358662362, + "rescued": 628, + "harmed": 409, + "missed_rescue": 158, + "avoided_harm": 66, + "selected_both_correct": 1209, + "selected_both_wrong": 106 + }, + { + "budget": 0.85, + "threshold": -0.15291842818260193, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.1711356788768453, + "rescued": 639, + "harmed": 411, + "missed_rescue": 147, + "avoided_harm": 64, + "selected_both_correct": 1218, + "selected_both_wrong": 112 + }, + { + "budget": 0.86, + "threshold": -0.15889839828014374, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.17267738998377383, + "rescued": 647, + "harmed": 416, + "missed_rescue": 139, + "avoided_harm": 59, + "selected_both_correct": 1230, + "selected_both_wrong": 115 + }, + { + "budget": 0.87, + "threshold": -0.16629797220230103, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.17579084329823536, + "rescued": 656, + "harmed": 422, + "missed_rescue": 130, + "avoided_harm": 53, + "selected_both_correct": 1241, + "selected_both_wrong": 117 + }, + { + "budget": 0.88, + "threshold": -0.17295491695404053, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.17752359710692284, + "rescued": 666, + "harmed": 427, + "missed_rescue": 120, + "avoided_harm": 48, + "selected_both_correct": 1253, + "selected_both_wrong": 118 + }, + { + "budget": 0.89, + "threshold": -0.18053314089775085, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.18131944434726105, + "rescued": 676, + "harmed": 431, + "missed_rescue": 110, + "avoided_harm": 44, + "selected_both_correct": 1266, + "selected_both_wrong": 119 + }, + { + "budget": 0.9, + "threshold": -0.19442276656627655, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.18427430914771328, + "rescued": 688, + "harmed": 433, + "missed_rescue": 98, + "avoided_harm": 42, + "selected_both_correct": 1277, + "selected_both_wrong": 122 + }, + { + "budget": 0.91, + "threshold": -0.20900504291057587, + "operator": "ge", + "actual_fraction": 0.9087018544935807, + "metric": 0.18611646191425543, + "rescued": 697, + "harmed": 437, + "missed_rescue": 89, + "avoided_harm": 38, + "selected_both_correct": 1289, + "selected_both_wrong": 125 + }, + { + "budget": 0.92, + "threshold": -0.22383525967597961, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.1910397694023846, + "rescued": 707, + "harmed": 440, + "missed_rescue": 79, + "avoided_harm": 35, + "selected_both_correct": 1299, + "selected_both_wrong": 130 + }, + { + "budget": 0.93, + "threshold": -0.25952714681625366, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.19063222783395556, + "rescued": 717, + "harmed": 447, + "missed_rescue": 69, + "avoided_harm": 28, + "selected_both_correct": 1308, + "selected_both_wrong": 132 + }, + { + "budget": 0.94, + "threshold": -0.2804492115974426, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.19317109979249197, + "rescued": 726, + "harmed": 451, + "missed_rescue": 60, + "avoided_harm": 24, + "selected_both_correct": 1318, + "selected_both_wrong": 137 + }, + { + "budget": 0.95, + "threshold": -0.2984963357448578, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.19747037448756763, + "rescued": 738, + "harmed": 453, + "missed_rescue": 48, + "avoided_harm": 22, + "selected_both_correct": 1329, + "selected_both_wrong": 140 + }, + { + "budget": 0.96, + "threshold": -0.32223379611968994, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.2013335009829101, + "rescued": 748, + "harmed": 455, + "missed_rescue": 38, + "avoided_harm": 20, + "selected_both_correct": 1343, + "selected_both_wrong": 142 + }, + { + "budget": 0.97, + "threshold": -0.36503666639328003, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.2032615598092342, + "rescued": 758, + "harmed": 460, + "missed_rescue": 28, + "avoided_harm": 15, + "selected_both_correct": 1355, + "selected_both_wrong": 143 + }, + { + "budget": 0.98, + "threshold": -0.4221048653125763, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.20572745422879232, + "rescued": 770, + "harmed": 465, + "missed_rescue": 16, + "avoided_harm": 10, + "selected_both_correct": 1366, + "selected_both_wrong": 143 + }, + { + "budget": 0.99, + "threshold": -0.44162338972091675, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.20340063714272894, + "rescued": 778, + "harmed": 472, + "missed_rescue": 8, + "avoided_harm": 3, + "selected_both_correct": 1374, + "selected_both_wrong": 148 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.20666016863629594, + "rescued": 786, + "harmed": 475, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 1391, + "selected_both_wrong": 152 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.07887397871043929, + "rescued": 0, + "harmed": 0, + "missed_rescue": 3314, + "avoided_harm": 1945, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.021043073385953903, + "operator": "ge", + "actual_fraction": 0.007989347536617843, + "metric": 0.0788343605891772, + "rescued": 0, + "harmed": 21, + "missed_rescue": 3314, + "avoided_harm": 1924, + "selected_both_correct": 75, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -0.022327054291963577, + "operator": "ge", + "actual_fraction": 0.016644474034620507, + "metric": 0.07894526738759718, + "rescued": 1, + "harmed": 39, + "missed_rescue": 3313, + "avoided_harm": 1906, + "selected_both_correct": 160, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -0.024909960106015205, + "operator": "ge", + "actual_fraction": 0.0296271637816245, + "metric": 0.07886280908239668, + "rescued": 1, + "harmed": 69, + "missed_rescue": 3313, + "avoided_harm": 1876, + "selected_both_correct": 286, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": -0.027814723551273346, + "operator": "ge", + "actual_fraction": 0.046937416777629824, + "metric": 0.07876261254060994, + "rescued": 1, + "harmed": 102, + "missed_rescue": 3313, + "avoided_harm": 1843, + "selected_both_correct": 461, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": -0.028914429247379303, + "operator": "ge", + "actual_fraction": 0.05326231691078562, + "metric": 0.07922967968226176, + "rescued": 2, + "harmed": 110, + "missed_rescue": 3312, + "avoided_harm": 1835, + "selected_both_correct": 528, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": -0.030170045793056488, + "operator": "ge", + "actual_fraction": 0.05992010652463382, + "metric": 0.07917808767949343, + "rescued": 2, + "harmed": 124, + "missed_rescue": 3312, + "avoided_harm": 1821, + "selected_both_correct": 594, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": -0.03123854100704193, + "operator": "ge", + "actual_fraction": 0.06724367509986684, + "metric": 0.08652168148030992, + "rescued": 5, + "harmed": 139, + "missed_rescue": 3309, + "avoided_harm": 1806, + "selected_both_correct": 664, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": -0.03410686179995537, + "operator": "ge", + "actual_fraction": 0.08721704394141146, + "metric": 0.08573997960186801, + "rescued": 6, + "harmed": 182, + "missed_rescue": 3308, + "avoided_harm": 1763, + "selected_both_correct": 860, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": -0.03476816788315773, + "operator": "ge", + "actual_fraction": 0.09287616511318242, + "metric": 0.08533855094545621, + "rescued": 6, + "harmed": 200, + "missed_rescue": 3308, + "avoided_harm": 1745, + "selected_both_correct": 910, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": -0.0365506075322628, + "operator": "ge", + "actual_fraction": 0.10785619174434088, + "metric": 0.08462726172642225, + "rescued": 6, + "harmed": 235, + "missed_rescue": 3308, + "avoided_harm": 1710, + "selected_both_correct": 1054, + "selected_both_wrong": 1 + }, + { + "budget": 0.11, + "threshold": -0.03818900138139725, + "operator": "ge", + "actual_fraction": 0.11784287616511319, + "metric": 0.08462444866867365, + "rescued": 7, + "harmed": 256, + "missed_rescue": 3307, + "avoided_harm": 1689, + "selected_both_correct": 1151, + "selected_both_wrong": 2 + }, + { + "budget": 0.12, + "threshold": -0.039673544466495514, + "operator": "ge", + "actual_fraction": 0.12716378162450068, + "metric": 0.08431158675236691, + "rescued": 7, + "harmed": 279, + "missed_rescue": 3307, + "avoided_harm": 1666, + "selected_both_correct": 1240, + "selected_both_wrong": 2 + }, + { + "budget": 0.13, + "threshold": -0.04085410386323929, + "operator": "ge", + "actual_fraction": 0.13648468708388814, + "metric": 0.0840276750632089, + "rescued": 7, + "harmed": 300, + "missed_rescue": 3307, + "avoided_harm": 1645, + "selected_both_correct": 1331, + "selected_both_wrong": 2 + }, + { + "budget": 0.14, + "threshold": -0.04204101860523224, + "operator": "ge", + "actual_fraction": 0.14613848202396804, + "metric": 0.08385470737270147, + "rescued": 7, + "harmed": 319, + "missed_rescue": 3307, + "avoided_harm": 1626, + "selected_both_correct": 1428, + "selected_both_wrong": 2 + }, + { + "budget": 0.15, + "threshold": -0.04278091713786125, + "operator": "ge", + "actual_fraction": 0.1514647137150466, + "metric": 0.08370670942943897, + "rescued": 7, + "harmed": 332, + "missed_rescue": 3307, + "avoided_harm": 1613, + "selected_both_correct": 1479, + "selected_both_wrong": 2 + }, + { + "budget": 0.16, + "threshold": -0.04322759434580803, + "operator": "ge", + "actual_fraction": 0.15579227696404793, + "metric": 0.08361544458700533, + "rescued": 7, + "harmed": 342, + "missed_rescue": 3307, + "avoided_harm": 1603, + "selected_both_correct": 1521, + "selected_both_wrong": 2 + }, + { + "budget": 0.17, + "threshold": -0.04394488036632538, + "operator": "ge", + "actual_fraction": 0.1617842876165113, + "metric": 0.08351220434393924, + "rescued": 7, + "harmed": 352, + "missed_rescue": 3307, + "avoided_harm": 1593, + "selected_both_correct": 1583, + "selected_both_wrong": 2 + }, + { + "budget": 0.18, + "threshold": -0.04530126228928566, + "operator": "ge", + "actual_fraction": 0.17177097203728361, + "metric": 0.08333120709231198, + "rescued": 7, + "harmed": 379, + "missed_rescue": 3307, + "avoided_harm": 1566, + "selected_both_correct": 1676, + "selected_both_wrong": 2 + }, + { + "budget": 0.19, + "threshold": -0.04661082476377487, + "operator": "ge", + "actual_fraction": 0.1810918774966711, + "metric": 0.08341398307559608, + "rescued": 8, + "harmed": 398, + "missed_rescue": 3306, + "avoided_harm": 1547, + "selected_both_correct": 1768, + "selected_both_wrong": 2 + }, + { + "budget": 0.2, + "threshold": -0.04730507358908653, + "operator": "ge", + "actual_fraction": 0.1870838881491345, + "metric": 0.08516887352267764, + "rescued": 9, + "harmed": 415, + "missed_rescue": 3305, + "avoided_harm": 1530, + "selected_both_correct": 1821, + "selected_both_wrong": 3 + }, + { + "budget": 0.21, + "threshold": -0.048246145248413086, + "operator": "ge", + "actual_fraction": 0.19640479360852198, + "metric": 0.08675280750372345, + "rescued": 10, + "harmed": 435, + "missed_rescue": 3304, + "avoided_harm": 1510, + "selected_both_correct": 1911, + "selected_both_wrong": 4 + }, + { + "budget": 0.22, + "threshold": -0.04900278151035309, + "operator": "ge", + "actual_fraction": 0.20838881491344874, + "metric": 0.08619173713061849, + "rescued": 10, + "harmed": 468, + "missed_rescue": 3304, + "avoided_harm": 1477, + "selected_both_correct": 2019, + "selected_both_wrong": 7 + }, + { + "budget": 0.23, + "threshold": -0.050042688846588135, + "operator": "ge", + "actual_fraction": 0.21471371504660453, + "metric": 0.08594774173844119, + "rescued": 10, + "harmed": 483, + "missed_rescue": 3304, + "avoided_harm": 1462, + "selected_both_correct": 2080, + "selected_both_wrong": 7 + }, + { + "budget": 0.24, + "threshold": -0.050901688635349274, + "operator": "ge", + "actual_fraction": 0.22536617842876164, + "metric": 0.08804054363419953, + "rescued": 41, + "harmed": 502, + "missed_rescue": 3273, + "avoided_harm": 1443, + "selected_both_correct": 2154, + "selected_both_wrong": 11 + }, + { + "budget": 0.25, + "threshold": -0.05161922052502632, + "operator": "ge", + "actual_fraction": 0.23135818908122505, + "metric": 0.08818772258585045, + "rescued": 60, + "harmed": 517, + "missed_rescue": 3254, + "avoided_harm": 1428, + "selected_both_correct": 2192, + "selected_both_wrong": 11 + }, + { + "budget": 0.26, + "threshold": -0.05245445668697357, + "operator": "ge", + "actual_fraction": 0.237017310252996, + "metric": 0.08841839409893854, + "rescued": 76, + "harmed": 525, + "missed_rescue": 3238, + "avoided_harm": 1420, + "selected_both_correct": 2235, + "selected_both_wrong": 12 + }, + { + "budget": 0.27, + "threshold": -0.053079016506671906, + "operator": "ge", + "actual_fraction": 0.24167776298268975, + "metric": 0.08850628284208313, + "rescued": 90, + "harmed": 536, + "missed_rescue": 3224, + "avoided_harm": 1409, + "selected_both_correct": 2265, + "selected_both_wrong": 13 + }, + { + "budget": 0.28, + "threshold": -0.053604189306497574, + "operator": "ge", + "actual_fraction": 0.2470039946737683, + "metric": 0.0886323434450378, + "rescued": 106, + "harmed": 548, + "missed_rescue": 3208, + "avoided_harm": 1397, + "selected_both_correct": 2301, + "selected_both_wrong": 13 + }, + { + "budget": 0.29, + "threshold": -0.05505518987774849, + "operator": "ge", + "actual_fraction": 0.2623169107856192, + "metric": 0.09067349863003854, + "rescued": 158, + "harmed": 582, + "missed_rescue": 3156, + "avoided_harm": 1363, + "selected_both_correct": 2396, + "selected_both_wrong": 16 + }, + { + "budget": 0.3, + "threshold": -0.05605518817901611, + "operator": "ge", + "actual_fraction": 0.27463382157123833, + "metric": 0.09250161598017743, + "rescued": 201, + "harmed": 607, + "missed_rescue": 3113, + "avoided_harm": 1338, + "selected_both_correct": 2469, + "selected_both_wrong": 23 + }, + { + "budget": 0.31, + "threshold": -0.056986160576343536, + "operator": "ge", + "actual_fraction": 0.28462050599201066, + "metric": 0.09322240603889413, + "rescued": 233, + "harmed": 625, + "missed_rescue": 3081, + "avoided_harm": 1320, + "selected_both_correct": 2537, + "selected_both_wrong": 25 + }, + { + "budget": 0.32, + "threshold": -0.0582931749522686, + "operator": "ge", + "actual_fraction": 0.29427430093209056, + "metric": 0.09484741449397432, + "rescued": 264, + "harmed": 642, + "missed_rescue": 3050, + "avoided_harm": 1303, + "selected_both_correct": 2598, + "selected_both_wrong": 32 + }, + { + "budget": 0.33, + "threshold": -0.0596650205552578, + "operator": "ge", + "actual_fraction": 0.3062583222370173, + "metric": 0.09509370405935859, + "rescued": 304, + "harmed": 671, + "missed_rescue": 3010, + "avoided_harm": 1274, + "selected_both_correct": 2671, + "selected_both_wrong": 34 + }, + { + "budget": 0.34, + "threshold": -0.060549296438694, + "operator": "ge", + "actual_fraction": 0.3132490013315579, + "metric": 0.09537687922458282, + "rescued": 331, + "harmed": 687, + "missed_rescue": 2983, + "avoided_harm": 1258, + "selected_both_correct": 2709, + "selected_both_wrong": 37 + }, + { + "budget": 0.35, + "threshold": -0.06178095191717148, + "operator": "ge", + "actual_fraction": 0.3189081225033289, + "metric": 0.09518164881276557, + "rescued": 349, + "harmed": 703, + "missed_rescue": 2965, + "avoided_harm": 1242, + "selected_both_correct": 2741, + "selected_both_wrong": 39 + }, + { + "budget": 0.36, + "threshold": -0.06328777223825455, + "operator": "ge", + "actual_fraction": 0.3318908122503329, + "metric": 0.09579948445467232, + "rescued": 389, + "harmed": 731, + "missed_rescue": 2925, + "avoided_harm": 1214, + "selected_both_correct": 2819, + "selected_both_wrong": 49 + }, + { + "budget": 0.37, + "threshold": -0.0647548958659172, + "operator": "ge", + "actual_fraction": 0.3418774966711052, + "metric": 0.09604832793526258, + "rescued": 424, + "harmed": 753, + "missed_rescue": 2890, + "avoided_harm": 1192, + "selected_both_correct": 2877, + "selected_both_wrong": 54 + }, + { + "budget": 0.38, + "threshold": -0.06637481600046158, + "operator": "ge", + "actual_fraction": 0.35352862849533956, + "metric": 0.09866030704730355, + "rescued": 472, + "harmed": 774, + "missed_rescue": 2842, + "avoided_harm": 1171, + "selected_both_correct": 2942, + "selected_both_wrong": 60 + }, + { + "budget": 0.39, + "threshold": -0.06774259358644485, + "operator": "ge", + "actual_fraction": 0.3641810918774967, + "metric": 0.10022118439970583, + "rescued": 514, + "harmed": 797, + "missed_rescue": 2800, + "avoided_harm": 1148, + "selected_both_correct": 3000, + "selected_both_wrong": 65 + }, + { + "budget": 0.4, + "threshold": -0.06843380630016327, + "operator": "ge", + "actual_fraction": 0.3698402130492676, + "metric": 0.10099642512611362, + "rescued": 537, + "harmed": 810, + "missed_rescue": 2777, + "avoided_harm": 1135, + "selected_both_correct": 3027, + "selected_both_wrong": 70 + }, + { + "budget": 0.41, + "threshold": -0.0694691464304924, + "operator": "ge", + "actual_fraction": 0.3801597869507324, + "metric": 0.10286568189357695, + "rescued": 582, + "harmed": 836, + "missed_rescue": 2732, + "avoided_harm": 1109, + "selected_both_correct": 3076, + "selected_both_wrong": 74 + }, + { + "budget": 0.42, + "threshold": -0.07112707942724228, + "operator": "ge", + "actual_fraction": 0.3938082556591212, + "metric": 0.1036415927515812, + "rescued": 630, + "harmed": 864, + "missed_rescue": 2684, + "avoided_harm": 1081, + "selected_both_correct": 3152, + "selected_both_wrong": 86 + }, + { + "budget": 0.43, + "threshold": -0.07192998379468918, + "operator": "ge", + "actual_fraction": 0.3988015978695073, + "metric": 0.10383106693904906, + "rescued": 650, + "harmed": 874, + "missed_rescue": 2664, + "avoided_harm": 1071, + "selected_both_correct": 3180, + "selected_both_wrong": 88 + }, + { + "budget": 0.44, + "threshold": -0.07266208529472351, + "operator": "ge", + "actual_fraction": 0.40279627163781623, + "metric": 0.10497587071399195, + "rescued": 665, + "harmed": 882, + "missed_rescue": 2649, + "avoided_harm": 1063, + "selected_both_correct": 3204, + "selected_both_wrong": 89 + }, + { + "budget": 0.45, + "threshold": -0.0744253620505333, + "operator": "ge", + "actual_fraction": 0.41711051930758986, + "metric": 0.10625030009652583, + "rescued": 721, + "harmed": 910, + "missed_rescue": 2593, + "avoided_harm": 1035, + "selected_both_correct": 3282, + "selected_both_wrong": 99 + }, + { + "budget": 0.46, + "threshold": -0.07532940804958344, + "operator": "ge", + "actual_fraction": 0.42243675099866845, + "metric": 0.10676350886210961, + "rescued": 745, + "harmed": 919, + "missed_rescue": 2569, + "avoided_harm": 1026, + "selected_both_correct": 3307, + "selected_both_wrong": 105 + }, + { + "budget": 0.47, + "threshold": -0.0770750343799591, + "operator": "ge", + "actual_fraction": 0.4380825565912117, + "metric": 0.10893787008842995, + "rescued": 817, + "harmed": 955, + "missed_rescue": 2497, + "avoided_harm": 990, + "selected_both_correct": 3377, + "selected_both_wrong": 115 + }, + { + "budget": 0.48, + "threshold": -0.07818037271499634, + "operator": "ge", + "actual_fraction": 0.44573901464713717, + "metric": 0.10946867779982243, + "rescued": 849, + "harmed": 973, + "missed_rescue": 2465, + "avoided_harm": 972, + "selected_both_correct": 3412, + "selected_both_wrong": 122 + }, + { + "budget": 0.49, + "threshold": -0.07998209446668625, + "operator": "ge", + "actual_fraction": 0.45905459387483355, + "metric": 0.11125477030640087, + "rescued": 900, + "harmed": 995, + "missed_rescue": 2414, + "avoided_harm": 950, + "selected_both_correct": 3484, + "selected_both_wrong": 137 + }, + { + "budget": 0.5, + "threshold": -0.08124949783086777, + "operator": "ge", + "actual_fraction": 0.46937416777629826, + "metric": 0.1119628778237788, + "rescued": 946, + "harmed": 1016, + "missed_rescue": 2368, + "avoided_harm": 929, + "selected_both_correct": 3537, + "selected_both_wrong": 141 + }, + { + "budget": 0.51, + "threshold": -0.08398465067148209, + "operator": "ge", + "actual_fraction": 0.49467376830892146, + "metric": 0.11739090876226205, + "rescued": 1064, + "harmed": 1070, + "missed_rescue": 2250, + "avoided_harm": 875, + "selected_both_correct": 3653, + "selected_both_wrong": 157 + }, + { + "budget": 0.52, + "threshold": -0.08537434041500092, + "operator": "ge", + "actual_fraction": 0.5056591211717709, + "metric": 0.11981696243623577, + "rescued": 1112, + "harmed": 1093, + "missed_rescue": 2202, + "avoided_harm": 852, + "selected_both_correct": 3704, + "selected_both_wrong": 167 + }, + { + "budget": 0.53, + "threshold": -0.08664409816265106, + "operator": "ge", + "actual_fraction": 0.5159786950732357, + "metric": 0.12050661028396976, + "rescued": 1152, + "harmed": 1116, + "missed_rescue": 2162, + "avoided_harm": 829, + "selected_both_correct": 3756, + "selected_both_wrong": 176 + }, + { + "budget": 0.54, + "threshold": -0.08777308464050293, + "operator": "ge", + "actual_fraction": 0.524300932090546, + "metric": 0.12109159247774551, + "rescued": 1190, + "harmed": 1137, + "missed_rescue": 2124, + "avoided_harm": 808, + "selected_both_correct": 3793, + "selected_both_wrong": 180 + }, + { + "budget": 0.55, + "threshold": -0.08980843424797058, + "operator": "ge", + "actual_fraction": 0.5386151797603196, + "metric": 0.12283117300011699, + "rescued": 1260, + "harmed": 1164, + "missed_rescue": 2054, + "avoided_harm": 781, + "selected_both_correct": 3860, + "selected_both_wrong": 188 + }, + { + "budget": 0.56, + "threshold": -0.0906188115477562, + "operator": "ge", + "actual_fraction": 0.546271637816245, + "metric": 0.12427034914228642, + "rescued": 1293, + "harmed": 1180, + "missed_rescue": 2021, + "avoided_harm": 765, + "selected_both_correct": 3897, + "selected_both_wrong": 194 + }, + { + "budget": 0.57, + "threshold": -0.09160211682319641, + "operator": "ge", + "actual_fraction": 0.5555925432756325, + "metric": 0.12600739832532146, + "rescued": 1339, + "harmed": 1194, + "missed_rescue": 1975, + "avoided_harm": 751, + "selected_both_correct": 3940, + "selected_both_wrong": 203 + }, + { + "budget": 0.58, + "threshold": -0.09283093363046646, + "operator": "ge", + "actual_fraction": 0.568242343541944, + "metric": 0.12948657153370394, + "rescued": 1401, + "harmed": 1224, + "missed_rescue": 1913, + "avoided_harm": 721, + "selected_both_correct": 3996, + "selected_both_wrong": 207 + }, + { + "budget": 0.59, + "threshold": -0.0941123366355896, + "operator": "ge", + "actual_fraction": 0.5778961384820239, + "metric": 0.1321361114270833, + "rescued": 1447, + "harmed": 1245, + "missed_rescue": 1867, + "avoided_harm": 700, + "selected_both_correct": 4036, + "selected_both_wrong": 216 + }, + { + "budget": 0.6, + "threshold": -0.09614869952201843, + "operator": "ge", + "actual_fraction": 0.5878828229027963, + "metric": 0.13359562560258884, + "rescued": 1495, + "harmed": 1262, + "missed_rescue": 1819, + "avoided_harm": 683, + "selected_both_correct": 4083, + "selected_both_wrong": 224 + }, + { + "budget": 0.61, + "threshold": -0.09772664308547974, + "operator": "ge", + "actual_fraction": 0.5975366178428761, + "metric": 0.135418307343696, + "rescued": 1537, + "harmed": 1279, + "missed_rescue": 1777, + "avoided_harm": 666, + "selected_both_correct": 4133, + "selected_both_wrong": 231 + }, + { + "budget": 0.62, + "threshold": -0.09911532700061798, + "operator": "ge", + "actual_fraction": 0.6081890812250332, + "metric": 0.13720391892557468, + "rescued": 1585, + "harmed": 1294, + "missed_rescue": 1729, + "avoided_harm": 651, + "selected_both_correct": 4189, + "selected_both_wrong": 240 + }, + { + "budget": 0.63, + "threshold": -0.10007988661527634, + "operator": "ge", + "actual_fraction": 0.6161784287616512, + "metric": 0.13839536227953134, + "rescued": 1625, + "harmed": 1312, + "missed_rescue": 1689, + "avoided_harm": 633, + "selected_both_correct": 4222, + "selected_both_wrong": 245 + }, + { + "budget": 0.64, + "threshold": -0.10177135467529297, + "operator": "ge", + "actual_fraction": 0.6278295605858855, + "metric": 0.14186473384550358, + "rescued": 1685, + "harmed": 1327, + "missed_rescue": 1629, + "avoided_harm": 618, + "selected_both_correct": 4280, + "selected_both_wrong": 252 + }, + { + "budget": 0.65, + "threshold": -0.10395076125860214, + "operator": "ge", + "actual_fraction": 0.6451398135818908, + "metric": 0.14514480336324412, + "rescued": 1771, + "harmed": 1357, + "missed_rescue": 1543, + "avoided_harm": 588, + "selected_both_correct": 4362, + "selected_both_wrong": 262 + }, + { + "budget": 0.66, + "threshold": -0.10524322092533112, + "operator": "ge", + "actual_fraction": 0.6491344873501997, + "metric": 0.14705419026423194, + "rescued": 1792, + "harmed": 1362, + "missed_rescue": 1522, + "avoided_harm": 583, + "selected_both_correct": 4381, + "selected_both_wrong": 265 + }, + { + "budget": 0.67, + "threshold": -0.10689426958560944, + "operator": "ge", + "actual_fraction": 0.6601198402130493, + "metric": 0.14778857092985256, + "rescued": 1842, + "harmed": 1385, + "missed_rescue": 1472, + "avoided_harm": 560, + "selected_both_correct": 4428, + "selected_both_wrong": 277 + }, + { + "budget": 0.68, + "threshold": -0.11014843732118607, + "operator": "ge", + "actual_fraction": 0.6754327563249002, + "metric": 0.15144863687072044, + "rescued": 1911, + "harmed": 1408, + "missed_rescue": 1403, + "avoided_harm": 537, + "selected_both_correct": 4509, + "selected_both_wrong": 288 + }, + { + "budget": 0.69, + "threshold": -0.11117545515298843, + "operator": "ge", + "actual_fraction": 0.6790945406125166, + "metric": 0.1527001767891735, + "rescued": 1931, + "harmed": 1414, + "missed_rescue": 1383, + "avoided_harm": 531, + "selected_both_correct": 4524, + "selected_both_wrong": 291 + }, + { + "budget": 0.7, + "threshold": -0.11230025440454483, + "operator": "ge", + "actual_fraction": 0.6870838881491345, + "metric": 0.15446609724321056, + "rescued": 1973, + "harmed": 1424, + "missed_rescue": 1341, + "avoided_harm": 521, + "selected_both_correct": 4561, + "selected_both_wrong": 298 + }, + { + "budget": 0.71, + "threshold": -0.11423606425523758, + "operator": "ge", + "actual_fraction": 0.6960719041278296, + "metric": 0.1566074758661257, + "rescued": 2016, + "harmed": 1440, + "missed_rescue": 1298, + "avoided_harm": 505, + "selected_both_correct": 4599, + "selected_both_wrong": 309 + }, + { + "budget": 0.72, + "threshold": -0.11647249013185501, + "operator": "ge", + "actual_fraction": 0.7053928095872171, + "metric": 0.15830843131436304, + "rescued": 2059, + "harmed": 1456, + "missed_rescue": 1255, + "avoided_harm": 489, + "selected_both_correct": 4643, + "selected_both_wrong": 318 + }, + { + "budget": 0.73, + "threshold": -0.11962417513132095, + "operator": "ge", + "actual_fraction": 0.7193741677762983, + "metric": 0.16212144588492078, + "rescued": 2121, + "harmed": 1481, + "missed_rescue": 1193, + "avoided_harm": 464, + "selected_both_correct": 4706, + "selected_both_wrong": 336 + }, + { + "budget": 0.74, + "threshold": -0.12126146256923676, + "operator": "ge", + "actual_fraction": 0.7250332889480693, + "metric": 0.16312718485088668, + "rescued": 2145, + "harmed": 1491, + "missed_rescue": 1169, + "avoided_harm": 454, + "selected_both_correct": 4732, + "selected_both_wrong": 344 + }, + { + "budget": 0.75, + "threshold": -0.1230982318520546, + "operator": "ge", + "actual_fraction": 0.7366844207723036, + "metric": 0.1660266588911306, + "rescued": 2198, + "harmed": 1515, + "missed_rescue": 1116, + "avoided_harm": 430, + "selected_both_correct": 4785, + "selected_both_wrong": 354 + }, + { + "budget": 0.76, + "threshold": -0.12517565488815308, + "operator": "ge", + "actual_fraction": 0.7446737683089214, + "metric": 0.16822905417177203, + "rescued": 2234, + "harmed": 1526, + "missed_rescue": 1080, + "avoided_harm": 419, + "selected_both_correct": 4828, + "selected_both_wrong": 360 + }, + { + "budget": 0.77, + "threshold": -0.12628257274627686, + "operator": "ge", + "actual_fraction": 0.7480026631158455, + "metric": 0.1691185459978786, + "rescued": 2251, + "harmed": 1531, + "missed_rescue": 1063, + "avoided_harm": 414, + "selected_both_correct": 4843, + "selected_both_wrong": 363 + }, + { + "budget": 0.78, + "threshold": -0.12975578010082245, + "operator": "ge", + "actual_fraction": 0.7603195739014648, + "metric": 0.17190478913940857, + "rescued": 2306, + "harmed": 1552, + "missed_rescue": 1008, + "avoided_harm": 393, + "selected_both_correct": 4897, + "selected_both_wrong": 381 + }, + { + "budget": 0.79, + "threshold": -0.13229140639305115, + "operator": "ge", + "actual_fraction": 0.7713049267643143, + "metric": 0.17440054880534744, + "rescued": 2355, + "harmed": 1568, + "missed_rescue": 959, + "avoided_harm": 377, + "selected_both_correct": 4952, + "selected_both_wrong": 393 + }, + { + "budget": 0.8, + "threshold": -0.1373685747385025, + "operator": "ge", + "actual_fraction": 0.7909454061251664, + "metric": 0.17894927236127425, + "rescued": 2455, + "harmed": 1608, + "missed_rescue": 859, + "avoided_harm": 337, + "selected_both_correct": 5030, + "selected_both_wrong": 411 + }, + { + "budget": 0.81, + "threshold": -0.1407174915075302, + "operator": "ge", + "actual_fraction": 0.8012649800266312, + "metric": 0.18049284683090172, + "rescued": 2502, + "harmed": 1631, + "missed_rescue": 812, + "avoided_harm": 314, + "selected_both_correct": 5069, + "selected_both_wrong": 426 + }, + { + "budget": 0.82, + "threshold": -0.1433146893978119, + "operator": "ge", + "actual_fraction": 0.8125832223701731, + "metric": 0.18188035357936716, + "rescued": 2556, + "harmed": 1658, + "missed_rescue": 758, + "avoided_harm": 287, + "selected_both_correct": 5115, + "selected_both_wrong": 435 + }, + { + "budget": 0.83, + "threshold": -0.1457100659608841, + "operator": "ge", + "actual_fraction": 0.8199067909454061, + "metric": 0.1842965007470308, + "rescued": 2593, + "harmed": 1670, + "missed_rescue": 721, + "avoided_harm": 275, + "selected_both_correct": 5145, + "selected_both_wrong": 444 + }, + { + "budget": 0.84, + "threshold": -0.148710235953331, + "operator": "ge", + "actual_fraction": 0.8242343541944075, + "metric": 0.18526487033383585, + "rescued": 2610, + "harmed": 1675, + "missed_rescue": 704, + "avoided_harm": 270, + "selected_both_correct": 5167, + "selected_both_wrong": 452 + }, + { + "budget": 0.85, + "threshold": -0.15291842818260193, + "operator": "ge", + "actual_fraction": 0.8338881491344874, + "metric": 0.18825160118942358, + "rescued": 2656, + "harmed": 1694, + "missed_rescue": 658, + "avoided_harm": 251, + "selected_both_correct": 5208, + "selected_both_wrong": 462 + }, + { + "budget": 0.86, + "threshold": -0.15889839828014374, + "operator": "ge", + "actual_fraction": 0.8458721704394141, + "metric": 0.19240881033397328, + "rescued": 2704, + "harmed": 1712, + "missed_rescue": 610, + "avoided_harm": 233, + "selected_both_correct": 5271, + "selected_both_wrong": 477 + }, + { + "budget": 0.87, + "threshold": -0.16629797220230103, + "operator": "ge", + "actual_fraction": 0.8585219707057257, + "metric": 0.1975333545503005, + "rescued": 2763, + "harmed": 1734, + "missed_rescue": 551, + "avoided_harm": 211, + "selected_both_correct": 5328, + "selected_both_wrong": 491 + }, + { + "budget": 0.88, + "threshold": -0.17295491695404053, + "operator": "ge", + "actual_fraction": 0.8655126498002663, + "metric": 0.19989071661116012, + "rescued": 2792, + "harmed": 1743, + "missed_rescue": 522, + "avoided_harm": 202, + "selected_both_correct": 5365, + "selected_both_wrong": 500 + }, + { + "budget": 0.89, + "threshold": -0.18053314089775085, + "operator": "ge", + "actual_fraction": 0.8738348868175766, + "metric": 0.20467943147275502, + "rescued": 2828, + "harmed": 1754, + "missed_rescue": 486, + "avoided_harm": 191, + "selected_both_correct": 5407, + "selected_both_wrong": 511 + }, + { + "budget": 0.9, + "threshold": -0.19442276656627655, + "operator": "ge", + "actual_fraction": 0.8878162450066578, + "metric": 0.20833827941063784, + "rescued": 2888, + "harmed": 1779, + "missed_rescue": 426, + "avoided_harm": 166, + "selected_both_correct": 5471, + "selected_both_wrong": 530 + }, + { + "budget": 0.91, + "threshold": -0.20900504291057587, + "operator": "ge", + "actual_fraction": 0.8961384820239681, + "metric": 0.2085245066088965, + "rescued": 2924, + "harmed": 1790, + "missed_rescue": 390, + "avoided_harm": 155, + "selected_both_correct": 5515, + "selected_both_wrong": 539 + }, + { + "budget": 0.92, + "threshold": -0.22383525967597961, + "operator": "ge", + "actual_fraction": 0.9027962716378163, + "metric": 0.2108748825898504, + "rescued": 2952, + "harmed": 1799, + "missed_rescue": 362, + "avoided_harm": 146, + "selected_both_correct": 5550, + "selected_both_wrong": 547 + }, + { + "budget": 0.93, + "threshold": -0.25952714681625366, + "operator": "ge", + "actual_fraction": 0.9184420772303595, + "metric": 0.21438099190536286, + "rescued": 3019, + "harmed": 1827, + "missed_rescue": 295, + "avoided_harm": 118, + "selected_both_correct": 5620, + "selected_both_wrong": 570 + }, + { + "budget": 0.94, + "threshold": -0.2804492115974426, + "operator": "ge", + "actual_fraction": 0.9257656458055925, + "metric": 0.21506366828770151, + "rescued": 3050, + "harmed": 1842, + "missed_rescue": 264, + "avoided_harm": 103, + "selected_both_correct": 5649, + "selected_both_wrong": 583 + }, + { + "budget": 0.95, + "threshold": -0.2984963357448578, + "operator": "ge", + "actual_fraction": 0.9320905459387483, + "metric": 0.21567531720794794, + "rescued": 3069, + "harmed": 1849, + "missed_rescue": 245, + "avoided_harm": 96, + "selected_both_correct": 5689, + "selected_both_wrong": 593 + }, + { + "budget": 0.96, + "threshold": -0.32223379611968994, + "operator": "ge", + "actual_fraction": 0.9410785619174434, + "metric": 0.21264209690310557, + "rescued": 3101, + "harmed": 1865, + "missed_rescue": 213, + "avoided_harm": 80, + "selected_both_correct": 5735, + "selected_both_wrong": 607 + }, + { + "budget": 0.97, + "threshold": -0.36503666639328003, + "operator": "ge", + "actual_fraction": 0.9573901464713716, + "metric": 0.21331564253317523, + "rescued": 3160, + "harmed": 1887, + "missed_rescue": 154, + "avoided_harm": 58, + "selected_both_correct": 5818, + "selected_both_wrong": 639 + }, + { + "budget": 0.98, + "threshold": -0.4221048653125763, + "operator": "ge", + "actual_fraction": 0.9790279627163782, + "metric": 0.21609512872414097, + "rescued": 3244, + "harmed": 1926, + "missed_rescue": 70, + "avoided_harm": 19, + "selected_both_correct": 5928, + "selected_both_wrong": 666 + }, + { + "budget": 0.99, + "threshold": -0.44162338972091675, + "operator": "ge", + "actual_fraction": 0.9860186418109188, + "metric": 0.21817095656956434, + "rescued": 3268, + "harmed": 1932, + "missed_rescue": 46, + "avoided_harm": 13, + "selected_both_correct": 5967, + "selected_both_wrong": 681 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.22361115727328065, + "rescued": 3314, + "harmed": 1945, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 6060, + "selected_both_wrong": 697 + } + ] + }, + "learned": { + "selected_budget_index": 78, + "validation_area_0_25": 0.11165309347543784, + "validation_point": { + "budget": 0.78, + "threshold": -2.93597412109375, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.21744085592769802, + "rescued": 784, + "harmed": 356, + "missed_rescue": 2, + "avoided_harm": 119, + "selected_both_correct": 894, + "selected_both_wrong": 150 + }, + "test_point": { + "budget": 0.78, + "threshold": -2.93597412109375, + "operator": "ge", + "actual_fraction": 0.7882822902796272, + "metric": 0.22996109134220227, + "rescued": 3304, + "harmed": 1470, + "missed_rescue": 10, + "avoided_harm": 475, + "selected_both_correct": 4007, + "selected_both_wrong": 691 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.07641657905799261, + "rescued": 0, + "harmed": 0, + "missed_rescue": 786, + "avoided_harm": 475, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.925830602645874, + "operator": "ge", + "actual_fraction": 0.009985734664764621, + "metric": 0.07541486249056822, + "rescued": 15, + "harmed": 4, + "missed_rescue": 771, + "avoided_harm": 471, + "selected_both_correct": 5, + "selected_both_wrong": 4 + }, + { + "budget": 0.02, + "threshold": 2.5482022762298584, + "operator": "ge", + "actual_fraction": 0.019971469329529243, + "metric": 0.0753394545529377, + "rescued": 30, + "harmed": 9, + "missed_rescue": 756, + "avoided_harm": 466, + "selected_both_correct": 11, + "selected_both_wrong": 6 + }, + { + "budget": 0.03, + "threshold": 2.464357376098633, + "operator": "ge", + "actual_fraction": 0.029957203994293864, + "metric": 0.07526581605528973, + "rescued": 38, + "harmed": 12, + "missed_rescue": 748, + "avoided_harm": 463, + "selected_both_correct": 22, + "selected_both_wrong": 12 + }, + { + "budget": 0.04, + "threshold": 2.3627471923828125, + "operator": "ge", + "actual_fraction": 0.039942938659058486, + "metric": 0.07528018831238462, + "rescued": 52, + "harmed": 14, + "missed_rescue": 734, + "avoided_harm": 461, + "selected_both_correct": 30, + "selected_both_wrong": 16 + }, + { + "budget": 0.05, + "threshold": 2.242785692214966, + "operator": "ge", + "actual_fraction": 0.04992867332382311, + "metric": 0.09891171926611136, + "rescued": 66, + "harmed": 15, + "missed_rescue": 720, + "avoided_harm": 460, + "selected_both_correct": 39, + "selected_both_wrong": 20 + }, + { + "budget": 0.06, + "threshold": 2.167039394378662, + "operator": "ge", + "actual_fraction": 0.05991440798858773, + "metric": 0.09601455602097679, + "rescued": 79, + "harmed": 19, + "missed_rescue": 707, + "avoided_harm": 456, + "selected_both_correct": 47, + "selected_both_wrong": 23 + }, + { + "budget": 0.07, + "threshold": 2.0983123779296875, + "operator": "ge", + "actual_fraction": 0.06990014265335236, + "metric": 0.1115182977050885, + "rescued": 93, + "harmed": 22, + "missed_rescue": 693, + "avoided_harm": 453, + "selected_both_correct": 56, + "selected_both_wrong": 25 + }, + { + "budget": 0.08, + "threshold": 2.0120444297790527, + "operator": "ge", + "actual_fraction": 0.07988587731811697, + "metric": 0.12543841855835197, + "rescued": 107, + "harmed": 24, + "missed_rescue": 679, + "avoided_harm": 451, + "selected_both_correct": 65, + "selected_both_wrong": 28 + }, + { + "budget": 0.09, + "threshold": 1.958116888999939, + "operator": "ge", + "actual_fraction": 0.0898716119828816, + "metric": 0.1249009918586407, + "rescued": 117, + "harmed": 26, + "missed_rescue": 669, + "avoided_harm": 449, + "selected_both_correct": 75, + "selected_both_wrong": 34 + }, + { + "budget": 0.1, + "threshold": 1.894916296005249, + "operator": "ge", + "actual_fraction": 0.09985734664764621, + "metric": 0.12435434631399415, + "rescued": 129, + "harmed": 27, + "missed_rescue": 657, + "avoided_harm": 448, + "selected_both_correct": 87, + "selected_both_wrong": 37 + }, + { + "budget": 0.11, + "threshold": 1.8710161447525024, + "operator": "ge", + "actual_fraction": 0.10984308131241084, + "metric": 0.12265151515151515, + "rescued": 141, + "harmed": 29, + "missed_rescue": 645, + "avoided_harm": 446, + "selected_both_correct": 97, + "selected_both_wrong": 41 + }, + { + "budget": 0.12, + "threshold": 1.831182599067688, + "operator": "ge", + "actual_fraction": 0.11982881597717546, + "metric": 0.11924907241143867, + "rescued": 152, + "harmed": 33, + "missed_rescue": 634, + "avoided_harm": 442, + "selected_both_correct": 107, + "selected_both_wrong": 44 + }, + { + "budget": 0.13, + "threshold": 1.7675763368606567, + "operator": "ge", + "actual_fraction": 0.12981455064194009, + "metric": 0.11806998490447855, + "rescued": 163, + "harmed": 35, + "missed_rescue": 623, + "avoided_harm": 440, + "selected_both_correct": 118, + "selected_both_wrong": 48 + }, + { + "budget": 0.14, + "threshold": 1.739072561264038, + "operator": "ge", + "actual_fraction": 0.1398002853067047, + "metric": 0.1179889554889555, + "rescued": 175, + "harmed": 36, + "missed_rescue": 611, + "avoided_harm": 439, + "selected_both_correct": 132, + "selected_both_wrong": 49 + }, + { + "budget": 0.15, + "threshold": 1.7081544399261475, + "operator": "ge", + "actual_fraction": 0.14978601997146934, + "metric": 0.11688888130416827, + "rescued": 187, + "harmed": 38, + "missed_rescue": 599, + "avoided_harm": 437, + "selected_both_correct": 143, + "selected_both_wrong": 52 + }, + { + "budget": 0.16, + "threshold": 1.6507465839385986, + "operator": "ge", + "actual_fraction": 0.15977175463623394, + "metric": 0.11355376456087356, + "rescued": 198, + "harmed": 43, + "missed_rescue": 588, + "avoided_harm": 432, + "selected_both_correct": 152, + "selected_both_wrong": 55 + }, + { + "budget": 0.17, + "threshold": 1.6030848026275635, + "operator": "ge", + "actual_fraction": 0.16975748930099857, + "metric": 0.11187710676430927, + "rescued": 213, + "harmed": 47, + "missed_rescue": 573, + "avoided_harm": 428, + "selected_both_correct": 159, + "selected_both_wrong": 57 + }, + { + "budget": 0.18, + "threshold": 1.5875954627990723, + "operator": "ge", + "actual_fraction": 0.1797432239657632, + "metric": 0.11155633561029354, + "rescued": 225, + "harmed": 52, + "missed_rescue": 561, + "avoided_harm": 423, + "selected_both_correct": 167, + "selected_both_wrong": 60 + }, + { + "budget": 0.19, + "threshold": 1.561208963394165, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.1192860883552164, + "rescued": 242, + "harmed": 56, + "missed_rescue": 544, + "avoided_harm": 419, + "selected_both_correct": 172, + "selected_both_wrong": 62 + }, + { + "budget": 0.2, + "threshold": 1.5279499292373657, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.11805768430313739, + "rescued": 257, + "harmed": 60, + "missed_rescue": 529, + "avoided_harm": 415, + "selected_both_correct": 178, + "selected_both_wrong": 65 + }, + { + "budget": 0.21, + "threshold": 1.4985991716384888, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.11773573369236094, + "rescued": 267, + "harmed": 64, + "missed_rescue": 519, + "avoided_harm": 411, + "selected_both_correct": 187, + "selected_both_wrong": 70 + }, + { + "budget": 0.22, + "threshold": 1.4575860500335693, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.12461831731812473, + "rescued": 282, + "harmed": 68, + "missed_rescue": 504, + "avoided_harm": 407, + "selected_both_correct": 193, + "selected_both_wrong": 73 + }, + { + "budget": 0.23, + "threshold": 1.4396969079971313, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.12389009614272777, + "rescued": 297, + "harmed": 72, + "missed_rescue": 489, + "avoided_harm": 403, + "selected_both_correct": 201, + "selected_both_wrong": 74 + }, + { + "budget": 0.24, + "threshold": 1.389216661453247, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.12388720644534598, + "rescued": 313, + "harmed": 76, + "missed_rescue": 473, + "avoided_harm": 399, + "selected_both_correct": 208, + "selected_both_wrong": 75 + }, + { + "budget": 0.25, + "threshold": 1.34909188747406, + "operator": "ge", + "actual_fraction": 0.24964336661911554, + "metric": 0.12368909332156336, + "rescued": 325, + "harmed": 80, + "missed_rescue": 461, + "avoided_harm": 395, + "selected_both_correct": 217, + "selected_both_wrong": 78 + }, + { + "budget": 0.26, + "threshold": 1.3260160684585571, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.12270781119465331, + "rescued": 337, + "harmed": 85, + "missed_rescue": 449, + "avoided_harm": 390, + "selected_both_correct": 224, + "selected_both_wrong": 82 + }, + { + "budget": 0.27, + "threshold": 1.2911567687988281, + "operator": "ge", + "actual_fraction": 0.26961483594864477, + "metric": 0.12317434049419759, + "rescued": 349, + "harmed": 89, + "missed_rescue": 437, + "avoided_harm": 386, + "selected_both_correct": 235, + "selected_both_wrong": 83 + }, + { + "budget": 0.28, + "threshold": 1.269096851348877, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.12356373808933148, + "rescued": 359, + "harmed": 91, + "missed_rescue": 427, + "avoided_harm": 384, + "selected_both_correct": 247, + "selected_both_wrong": 87 + }, + { + "budget": 0.29, + "threshold": 1.2518491744995117, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.12409499145304985, + "rescued": 376, + "harmed": 96, + "missed_rescue": 410, + "avoided_harm": 379, + "selected_both_correct": 253, + "selected_both_wrong": 87 + }, + { + "budget": 0.3, + "threshold": 1.2406281232833862, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.12413489787952423, + "rescued": 390, + "harmed": 101, + "missed_rescue": 396, + "avoided_harm": 374, + "selected_both_correct": 260, + "selected_both_wrong": 89 + }, + { + "budget": 0.31, + "threshold": 1.2157225608825684, + "operator": "ge", + "actual_fraction": 0.3095577746077033, + "metric": 0.1248690587107323, + "rescued": 399, + "harmed": 104, + "missed_rescue": 387, + "avoided_harm": 371, + "selected_both_correct": 272, + "selected_both_wrong": 93 + }, + { + "budget": 0.32, + "threshold": 1.2011241912841797, + "operator": "ge", + "actual_fraction": 0.3195435092724679, + "metric": 0.12629779568158003, + "rescued": 414, + "harmed": 108, + "missed_rescue": 372, + "avoided_harm": 367, + "selected_both_correct": 280, + "selected_both_wrong": 94 + }, + { + "budget": 0.33, + "threshold": 1.1690080165863037, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.12745336975017935, + "rescued": 429, + "harmed": 112, + "missed_rescue": 357, + "avoided_harm": 363, + "selected_both_correct": 288, + "selected_both_wrong": 95 + }, + { + "budget": 0.34, + "threshold": 1.1319067478179932, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.1272671679116844, + "rescued": 439, + "harmed": 118, + "missed_rescue": 347, + "avoided_harm": 357, + "selected_both_correct": 299, + "selected_both_wrong": 96 + }, + { + "budget": 0.35, + "threshold": 1.0817104578018188, + "operator": "ge", + "actual_fraction": 0.34950071326676174, + "metric": 0.12855190912892428, + "rescued": 454, + "harmed": 123, + "missed_rescue": 332, + "avoided_harm": 352, + "selected_both_correct": 307, + "selected_both_wrong": 96 + }, + { + "budget": 0.36, + "threshold": 1.050752878189087, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.12825262470290094, + "rescued": 460, + "harmed": 125, + "missed_rescue": 326, + "avoided_harm": 350, + "selected_both_correct": 322, + "selected_both_wrong": 101 + }, + { + "budget": 0.37, + "threshold": 1.0408045053482056, + "operator": "ge", + "actual_fraction": 0.369472182596291, + "metric": 0.12934025520218656, + "rescued": 471, + "harmed": 128, + "missed_rescue": 315, + "avoided_harm": 347, + "selected_both_correct": 333, + "selected_both_wrong": 104 + }, + { + "budget": 0.38, + "threshold": 1.0130165815353394, + "operator": "ge", + "actual_fraction": 0.37945791726105566, + "metric": 0.13450674940822724, + "rescued": 483, + "harmed": 133, + "missed_rescue": 303, + "avoided_harm": 342, + "selected_both_correct": 341, + "selected_both_wrong": 107 + }, + { + "budget": 0.39, + "threshold": 0.9877941012382507, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.1355578036304072, + "rescued": 494, + "harmed": 139, + "missed_rescue": 292, + "avoided_harm": 336, + "selected_both_correct": 350, + "selected_both_wrong": 109 + }, + { + "budget": 0.4, + "threshold": 0.9319636821746826, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.13647512823199684, + "rescued": 505, + "harmed": 144, + "missed_rescue": 281, + "avoided_harm": 331, + "selected_both_correct": 361, + "selected_both_wrong": 110 + }, + { + "budget": 0.41, + "threshold": 0.8980667591094971, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.14511107701914175, + "rescued": 518, + "harmed": 147, + "missed_rescue": 268, + "avoided_harm": 328, + "selected_both_correct": 370, + "selected_both_wrong": 113 + }, + { + "budget": 0.42, + "threshold": 0.8735002279281616, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.14651442929243694, + "rescued": 529, + "harmed": 151, + "missed_rescue": 257, + "avoided_harm": 324, + "selected_both_correct": 382, + "selected_both_wrong": 114 + }, + { + "budget": 0.43, + "threshold": 0.8648159503936768, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.1477145403615992, + "rescued": 536, + "harmed": 153, + "missed_rescue": 250, + "avoided_harm": 322, + "selected_both_correct": 398, + "selected_both_wrong": 117 + }, + { + "budget": 0.44, + "threshold": 0.8149229288101196, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.14729233225834817, + "rescued": 544, + "harmed": 163, + "missed_rescue": 242, + "avoided_harm": 312, + "selected_both_correct": 408, + "selected_both_wrong": 117 + }, + { + "budget": 0.45, + "threshold": 0.7909369468688965, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.1479564637873952, + "rescued": 555, + "harmed": 170, + "missed_rescue": 231, + "avoided_harm": 305, + "selected_both_correct": 417, + "selected_both_wrong": 118 + }, + { + "budget": 0.46, + "threshold": 0.7727278470993042, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.15343749622719718, + "rescued": 568, + "harmed": 174, + "missed_rescue": 218, + "avoided_harm": 301, + "selected_both_correct": 428, + "selected_both_wrong": 118 + }, + { + "budget": 0.47, + "threshold": 0.7434355020523071, + "operator": "ge", + "actual_fraction": 0.4693295292439372, + "metric": 0.15547035019822486, + "rescued": 580, + "harmed": 180, + "missed_rescue": 206, + "avoided_harm": 295, + "selected_both_correct": 437, + "selected_both_wrong": 119 + }, + { + "budget": 0.48, + "threshold": 0.7266849279403687, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.15604093183577644, + "rescued": 591, + "harmed": 186, + "missed_rescue": 195, + "avoided_harm": 289, + "selected_both_correct": 446, + "selected_both_wrong": 121 + }, + { + "budget": 0.49, + "threshold": 0.7074867486953735, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.15693317919851996, + "rescued": 599, + "harmed": 192, + "missed_rescue": 187, + "avoided_harm": 283, + "selected_both_correct": 459, + "selected_both_wrong": 122 + }, + { + "budget": 0.5, + "threshold": 0.6545048356056213, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.1569906914403694, + "rescued": 607, + "harmed": 198, + "missed_rescue": 179, + "avoided_harm": 277, + "selected_both_correct": 470, + "selected_both_wrong": 125 + }, + { + "budget": 0.51, + "threshold": 0.6124920845031738, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.15912480697707512, + "rescued": 617, + "harmed": 203, + "missed_rescue": 169, + "avoided_harm": 272, + "selected_both_correct": 482, + "selected_both_wrong": 126 + }, + { + "budget": 0.52, + "threshold": 0.5968987941741943, + "operator": "ge", + "actual_fraction": 0.5192582025677603, + "metric": 0.16496555704371488, + "rescued": 626, + "harmed": 208, + "missed_rescue": 160, + "avoided_harm": 267, + "selected_both_correct": 493, + "selected_both_wrong": 129 + }, + { + "budget": 0.53, + "threshold": 0.572314977645874, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.16656279736775093, + "rescued": 632, + "harmed": 212, + "missed_rescue": 154, + "avoided_harm": 263, + "selected_both_correct": 510, + "selected_both_wrong": 130 + }, + { + "budget": 0.54, + "threshold": 0.5496032238006592, + "operator": "ge", + "actual_fraction": 0.5392296718972895, + "metric": 0.16722662900576804, + "rescued": 640, + "harmed": 215, + "missed_rescue": 146, + "avoided_harm": 260, + "selected_both_correct": 527, + "selected_both_wrong": 130 + }, + { + "budget": 0.55, + "threshold": 0.5177246928215027, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.16673037512464242, + "rescued": 645, + "harmed": 220, + "missed_rescue": 141, + "avoided_harm": 255, + "selected_both_correct": 544, + "selected_both_wrong": 131 + }, + { + "budget": 0.56, + "threshold": 0.4967881739139557, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.16794760187373048, + "rescued": 651, + "harmed": 225, + "missed_rescue": 135, + "avoided_harm": 250, + "selected_both_correct": 560, + "selected_both_wrong": 132 + }, + { + "budget": 0.57, + "threshold": 0.4586660861968994, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.16955727662862627, + "rescued": 660, + "harmed": 232, + "missed_rescue": 126, + "avoided_harm": 243, + "selected_both_correct": 572, + "selected_both_wrong": 132 + }, + { + "budget": 0.58, + "threshold": 0.4310711920261383, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.17141912320483746, + "rescued": 668, + "harmed": 237, + "missed_rescue": 118, + "avoided_harm": 238, + "selected_both_correct": 586, + "selected_both_wrong": 133 + }, + { + "budget": 0.59, + "threshold": 0.39361393451690674, + "operator": "ge", + "actual_fraction": 0.5891583452211127, + "metric": 0.17332869887614413, + "rescued": 673, + "harmed": 239, + "missed_rescue": 113, + "avoided_harm": 236, + "selected_both_correct": 604, + "selected_both_wrong": 136 + }, + { + "budget": 0.6, + "threshold": 0.3438398838043213, + "operator": "ge", + "actual_fraction": 0.5991440798858774, + "metric": 0.17568053715139748, + "rescued": 680, + "harmed": 245, + "missed_rescue": 106, + "avoided_harm": 230, + "selected_both_correct": 619, + "selected_both_wrong": 136 + }, + { + "budget": 0.61, + "threshold": 0.31072938442230225, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.1769942338476258, + "rescued": 688, + "harmed": 251, + "missed_rescue": 98, + "avoided_harm": 224, + "selected_both_correct": 632, + "selected_both_wrong": 137 + }, + { + "budget": 0.62, + "threshold": 0.3001859486103058, + "operator": "ge", + "actual_fraction": 0.6191155492154066, + "metric": 0.17964116493528257, + "rescued": 695, + "harmed": 257, + "missed_rescue": 91, + "avoided_harm": 218, + "selected_both_correct": 647, + "selected_both_wrong": 137 + }, + { + "budget": 0.63, + "threshold": 0.2700633108615875, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.18283221498859456, + "rescued": 702, + "harmed": 260, + "missed_rescue": 84, + "avoided_harm": 215, + "selected_both_correct": 665, + "selected_both_wrong": 137 + }, + { + "budget": 0.64, + "threshold": 0.24940329790115356, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.1849286369075318, + "rescued": 708, + "harmed": 265, + "missed_rescue": 78, + "avoided_harm": 210, + "selected_both_correct": 681, + "selected_both_wrong": 138 + }, + { + "budget": 0.65, + "threshold": 0.21490943431854248, + "operator": "ge", + "actual_fraction": 0.6490727532097005, + "metric": 0.18706846418186626, + "rescued": 713, + "harmed": 270, + "missed_rescue": 73, + "avoided_harm": 205, + "selected_both_correct": 698, + "selected_both_wrong": 139 + }, + { + "budget": 0.66, + "threshold": 0.18740108609199524, + "operator": "ge", + "actual_fraction": 0.6590584878744651, + "metric": 0.18739860258634933, + "rescued": 719, + "harmed": 277, + "missed_rescue": 67, + "avoided_harm": 198, + "selected_both_correct": 713, + "selected_both_wrong": 139 + }, + { + "budget": 0.67, + "threshold": 0.17519652843475342, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.187542564740345, + "rescued": 724, + "harmed": 286, + "missed_rescue": 62, + "avoided_harm": 189, + "selected_both_correct": 726, + "selected_both_wrong": 140 + }, + { + "budget": 0.68, + "threshold": 0.13710853457450867, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.18951339267893497, + "rescued": 730, + "harmed": 292, + "missed_rescue": 56, + "avoided_harm": 183, + "selected_both_correct": 741, + "selected_both_wrong": 141 + }, + { + "budget": 0.69, + "threshold": 0.11163881421089172, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.19037528178251215, + "rescued": 737, + "harmed": 297, + "missed_rescue": 49, + "avoided_harm": 178, + "selected_both_correct": 756, + "selected_both_wrong": 142 + }, + { + "budget": 0.7, + "threshold": 0.0702638104557991, + "operator": "ge", + "actual_fraction": 0.6990014265335235, + "metric": 0.19310968449996166, + "rescued": 743, + "harmed": 302, + "missed_rescue": 43, + "avoided_harm": 173, + "selected_both_correct": 773, + "selected_both_wrong": 142 + }, + { + "budget": 0.71, + "threshold": 0.034758418798446655, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.1926297371779037, + "rescued": 747, + "harmed": 311, + "missed_rescue": 39, + "avoided_harm": 164, + "selected_both_correct": 785, + "selected_both_wrong": 145 + }, + { + "budget": 0.72, + "threshold": 0.01772361993789673, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.19654048968915047, + "rescued": 753, + "harmed": 318, + "missed_rescue": 33, + "avoided_harm": 157, + "selected_both_correct": 799, + "selected_both_wrong": 146 + }, + { + "budget": 0.73, + "threshold": -0.061358511447906494, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.19802517052328328, + "rescued": 759, + "harmed": 325, + "missed_rescue": 27, + "avoided_harm": 150, + "selected_both_correct": 814, + "selected_both_wrong": 146 + }, + { + "budget": 0.74, + "threshold": -0.10378703474998474, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.20243690966134922, + "rescued": 765, + "harmed": 331, + "missed_rescue": 21, + "avoided_harm": 144, + "selected_both_correct": 829, + "selected_both_wrong": 147 + }, + { + "budget": 0.75, + "threshold": -0.1483374536037445, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.2077706576158309, + "rescued": 771, + "harmed": 334, + "missed_rescue": 15, + "avoided_harm": 141, + "selected_both_correct": 847, + "selected_both_wrong": 148 + }, + { + "budget": 0.76, + "threshold": -0.20967978239059448, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.2105049786065888, + "rescued": 776, + "harmed": 341, + "missed_rescue": 10, + "avoided_harm": 134, + "selected_both_correct": 861, + "selected_both_wrong": 150 + }, + { + "budget": 0.77, + "threshold": -0.3404620289802551, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.2165636142111887, + "rescued": 782, + "harmed": 346, + "missed_rescue": 4, + "avoided_harm": 129, + "selected_both_correct": 878, + "selected_both_wrong": 150 + }, + { + "budget": 0.78, + "threshold": -2.93597412109375, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.21744085592769802, + "rescued": 784, + "harmed": 356, + "missed_rescue": 2, + "avoided_harm": 119, + "selected_both_correct": 894, + "selected_both_wrong": 150 + }, + { + "budget": 0.79, + "threshold": -3.1307332515716553, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.21697316885286808, + "rescued": 784, + "harmed": 361, + "missed_rescue": 2, + "avoided_harm": 114, + "selected_both_correct": 917, + "selected_both_wrong": 150 + }, + { + "budget": 0.8, + "threshold": -3.253781795501709, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.2163889764699013, + "rescued": 784, + "harmed": 368, + "missed_rescue": 2, + "avoided_harm": 107, + "selected_both_correct": 938, + "selected_both_wrong": 150 + }, + { + "budget": 0.81, + "threshold": -3.405195713043213, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.2160538237374819, + "rescued": 785, + "harmed": 372, + "missed_rescue": 1, + "avoided_harm": 103, + "selected_both_correct": 961, + "selected_both_wrong": 150 + }, + { + "budget": 0.82, + "threshold": -3.473080635070801, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.21573720397249807, + "rescued": 785, + "harmed": 376, + "missed_rescue": 1, + "avoided_harm": 99, + "selected_both_correct": 985, + "selected_both_wrong": 150 + }, + { + "budget": 0.83, + "threshold": -3.546388864517212, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.21447943089001706, + "rescued": 785, + "harmed": 383, + "missed_rescue": 1, + "avoided_harm": 92, + "selected_both_correct": 1006, + "selected_both_wrong": 150 + }, + { + "budget": 0.84, + "threshold": -3.6391501426696777, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.21408932713770384, + "rescued": 785, + "harmed": 388, + "missed_rescue": 1, + "avoided_harm": 87, + "selected_both_correct": 1028, + "selected_both_wrong": 151 + }, + { + "budget": 0.85, + "threshold": -3.692823886871338, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.2136228546342687, + "rescued": 785, + "harmed": 394, + "missed_rescue": 1, + "avoided_harm": 81, + "selected_both_correct": 1050, + "selected_both_wrong": 151 + }, + { + "budget": 0.86, + "threshold": -3.7845706939697266, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.21314663581866203, + "rescued": 785, + "harmed": 400, + "missed_rescue": 1, + "avoided_harm": 75, + "selected_both_correct": 1072, + "selected_both_wrong": 151 + }, + { + "budget": 0.87, + "threshold": -3.833963394165039, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.21192862721761446, + "rescued": 785, + "harmed": 407, + "missed_rescue": 1, + "avoided_harm": 68, + "selected_both_correct": 1093, + "selected_both_wrong": 151 + }, + { + "budget": 0.88, + "threshold": -3.90220308303833, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.21150119116605498, + "rescued": 785, + "harmed": 413, + "missed_rescue": 1, + "avoided_harm": 62, + "selected_both_correct": 1115, + "selected_both_wrong": 151 + }, + { + "budget": 0.89, + "threshold": -3.9384937286376953, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.21104766078247608, + "rescued": 785, + "harmed": 420, + "missed_rescue": 1, + "avoided_harm": 55, + "selected_both_correct": 1136, + "selected_both_wrong": 151 + }, + { + "budget": 0.9, + "threshold": -3.957301616668701, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.21082526833393522, + "rescued": 785, + "harmed": 427, + "missed_rescue": 1, + "avoided_harm": 48, + "selected_both_correct": 1157, + "selected_both_wrong": 151 + }, + { + "budget": 0.91, + "threshold": -4.0616135597229, + "operator": "ge", + "actual_fraction": 0.9087018544935807, + "metric": 0.21056993156544054, + "rescued": 785, + "harmed": 431, + "missed_rescue": 1, + "avoided_harm": 44, + "selected_both_correct": 1180, + "selected_both_wrong": 152 + }, + { + "budget": 0.92, + "threshold": -4.146303653717041, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.2102852588728341, + "rescued": 785, + "harmed": 436, + "missed_rescue": 1, + "avoided_harm": 39, + "selected_both_correct": 1203, + "selected_both_wrong": 152 + }, + { + "budget": 0.93, + "threshold": -4.184908390045166, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2081728703944853, + "rescued": 785, + "harmed": 441, + "missed_rescue": 1, + "avoided_harm": 34, + "selected_both_correct": 1226, + "selected_both_wrong": 152 + }, + { + "budget": 0.94, + "threshold": -4.23890495300293, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.20722233189273284, + "rescued": 785, + "harmed": 446, + "missed_rescue": 1, + "avoided_harm": 29, + "selected_both_correct": 1249, + "selected_both_wrong": 152 + }, + { + "budget": 0.95, + "threshold": -4.278663635253906, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.2068883987573162, + "rescued": 785, + "harmed": 452, + "missed_rescue": 1, + "avoided_harm": 23, + "selected_both_correct": 1271, + "selected_both_wrong": 152 + }, + { + "budget": 0.96, + "threshold": -4.356467247009277, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.20667850559726655, + "rescued": 785, + "harmed": 456, + "missed_rescue": 1, + "avoided_harm": 19, + "selected_both_correct": 1295, + "selected_both_wrong": 152 + }, + { + "budget": 0.97, + "threshold": -4.431670188903809, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.2080436220807073, + "rescued": 786, + "harmed": 461, + "missed_rescue": 0, + "avoided_harm": 14, + "selected_both_correct": 1317, + "selected_both_wrong": 152 + }, + { + "budget": 0.98, + "threshold": -4.527087688446045, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.2070558327567923, + "rescued": 786, + "harmed": 467, + "missed_rescue": 0, + "avoided_harm": 8, + "selected_both_correct": 1339, + "selected_both_wrong": 152 + }, + { + "budget": 0.99, + "threshold": -4.6759443283081055, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.2068560434867271, + "rescued": 786, + "harmed": 471, + "missed_rescue": 0, + "avoided_harm": 4, + "selected_both_correct": 1363, + "selected_both_wrong": 152 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.20666016863629594, + "rescued": 786, + "harmed": 475, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 1391, + "selected_both_wrong": 152 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.07887397871043929, + "rescued": 0, + "harmed": 0, + "missed_rescue": 3314, + "avoided_harm": 1945, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.925830602645874, + "operator": "ge", + "actual_fraction": 0.008655126498002663, + "metric": 0.08671603514515641, + "rescued": 52, + "harmed": 6, + "missed_rescue": 3262, + "avoided_harm": 1939, + "selected_both_correct": 34, + "selected_both_wrong": 12 + }, + { + "budget": 0.02, + "threshold": 2.5482022762298584, + "operator": "ge", + "actual_fraction": 0.02696404793608522, + "metric": 0.08587291690042786, + "rescued": 146, + "harmed": 22, + "missed_rescue": 3168, + "avoided_harm": 1923, + "selected_both_correct": 113, + "selected_both_wrong": 43 + }, + { + "budget": 0.03, + "threshold": 2.464357376098633, + "operator": "ge", + "actual_fraction": 0.03195739014647137, + "metric": 0.0859681459555215, + "rescued": 173, + "harmed": 26, + "missed_rescue": 3141, + "avoided_harm": 1919, + "selected_both_correct": 134, + "selected_both_wrong": 51 + }, + { + "budget": 0.04, + "threshold": 2.3627471923828125, + "operator": "ge", + "actual_fraction": 0.039280958721704395, + "metric": 0.09232483049090205, + "rescued": 213, + "harmed": 29, + "missed_rescue": 3101, + "avoided_harm": 1916, + "selected_both_correct": 164, + "selected_both_wrong": 66 + }, + { + "budget": 0.05, + "threshold": 2.242785692214966, + "operator": "ge", + "actual_fraction": 0.05292942743009321, + "metric": 0.0924366472698108, + "rescued": 288, + "harmed": 42, + "missed_rescue": 3026, + "avoided_harm": 1903, + "selected_both_correct": 220, + "selected_both_wrong": 86 + }, + { + "budget": 0.06, + "threshold": 2.167039394378662, + "operator": "ge", + "actual_fraction": 0.061917443408788284, + "metric": 0.09201910776844748, + "rescued": 327, + "harmed": 53, + "missed_rescue": 2987, + "avoided_harm": 1892, + "selected_both_correct": 262, + "selected_both_wrong": 102 + }, + { + "budget": 0.07, + "threshold": 2.0983123779296875, + "operator": "ge", + "actual_fraction": 0.06990679094540612, + "metric": 0.09227543447635411, + "rescued": 368, + "harmed": 59, + "missed_rescue": 2946, + "avoided_harm": 1886, + "selected_both_correct": 297, + "selected_both_wrong": 116 + }, + { + "budget": 0.08, + "threshold": 2.0120444297790527, + "operator": "ge", + "actual_fraction": 0.08189081225033289, + "metric": 0.09151815495760143, + "rescued": 424, + "harmed": 73, + "missed_rescue": 2890, + "avoided_harm": 1872, + "selected_both_correct": 351, + "selected_both_wrong": 136 + }, + { + "budget": 0.09, + "threshold": 1.958116888999939, + "operator": "ge", + "actual_fraction": 0.0888814913448735, + "metric": 0.09148238213705183, + "rescued": 461, + "harmed": 80, + "missed_rescue": 2853, + "avoided_harm": 1865, + "selected_both_correct": 379, + "selected_both_wrong": 148 + }, + { + "budget": 0.1, + "threshold": 1.894916296005249, + "operator": "ge", + "actual_fraction": 0.10252996005326231, + "metric": 0.0911755757978693, + "rescued": 529, + "harmed": 93, + "missed_rescue": 2785, + "avoided_harm": 1852, + "selected_both_correct": 442, + "selected_both_wrong": 168 + }, + { + "budget": 0.11, + "threshold": 1.8710161447525024, + "operator": "ge", + "actual_fraction": 0.10818908122503329, + "metric": 0.09813960112882866, + "rescued": 564, + "harmed": 100, + "missed_rescue": 2750, + "avoided_harm": 1845, + "selected_both_correct": 463, + "selected_both_wrong": 173 + }, + { + "budget": 0.12, + "threshold": 1.831182599067688, + "operator": "ge", + "actual_fraction": 0.11584553928095873, + "metric": 0.10154481780164515, + "rescued": 607, + "harmed": 110, + "missed_rescue": 2707, + "avoided_harm": 1835, + "selected_both_correct": 493, + "selected_both_wrong": 182 + }, + { + "budget": 0.13, + "threshold": 1.7675763368606567, + "operator": "ge", + "actual_fraction": 0.12949400798934754, + "metric": 0.10405610778139984, + "rescued": 675, + "harmed": 124, + "missed_rescue": 2639, + "avoided_harm": 1821, + "selected_both_correct": 554, + "selected_both_wrong": 203 + }, + { + "budget": 0.14, + "threshold": 1.739072561264038, + "operator": "ge", + "actual_fraction": 0.13448735019973368, + "metric": 0.1039060852075291, + "rescued": 702, + "harmed": 132, + "missed_rescue": 2612, + "avoided_harm": 1813, + "selected_both_correct": 569, + "selected_both_wrong": 213 + }, + { + "budget": 0.15, + "threshold": 1.7081544399261475, + "operator": "ge", + "actual_fraction": 0.14314247669773636, + "metric": 0.10361954885885534, + "rescued": 745, + "harmed": 141, + "missed_rescue": 2569, + "avoided_harm": 1804, + "selected_both_correct": 608, + "selected_both_wrong": 226 + }, + { + "budget": 0.16, + "threshold": 1.6507465839385986, + "operator": "ge", + "actual_fraction": 0.15579227696404793, + "metric": 0.10285750395231058, + "rescued": 810, + "harmed": 159, + "missed_rescue": 2504, + "avoided_harm": 1786, + "selected_both_correct": 661, + "selected_both_wrong": 242 + }, + { + "budget": 0.17, + "threshold": 1.6030848026275635, + "operator": "ge", + "actual_fraction": 0.1677762982689747, + "metric": 0.10214865606767878, + "rescued": 879, + "harmed": 183, + "missed_rescue": 2435, + "avoided_harm": 1762, + "selected_both_correct": 702, + "selected_both_wrong": 252 + }, + { + "budget": 0.18, + "threshold": 1.5875954627990723, + "operator": "ge", + "actual_fraction": 0.170439414114514, + "metric": 0.10200633269992133, + "rescued": 892, + "harmed": 188, + "missed_rescue": 2422, + "avoided_harm": 1757, + "selected_both_correct": 712, + "selected_both_wrong": 256 + }, + { + "budget": 0.19, + "threshold": 1.561208963394165, + "operator": "ge", + "actual_fraction": 0.17476697736351532, + "metric": 0.10413344947132269, + "rescued": 918, + "harmed": 196, + "missed_rescue": 2396, + "avoided_harm": 1749, + "selected_both_correct": 728, + "selected_both_wrong": 258 + }, + { + "budget": 0.2, + "threshold": 1.5279499292373657, + "operator": "ge", + "actual_fraction": 0.18508655126498003, + "metric": 0.10427942667511267, + "rescued": 971, + "harmed": 209, + "missed_rescue": 2343, + "avoided_harm": 1736, + "selected_both_correct": 773, + "selected_both_wrong": 271 + }, + { + "budget": 0.21, + "threshold": 1.4985991716384888, + "operator": "ge", + "actual_fraction": 0.19340878828229027, + "metric": 0.10432326854028226, + "rescued": 1011, + "harmed": 222, + "missed_rescue": 2303, + "avoided_harm": 1723, + "selected_both_correct": 811, + "selected_both_wrong": 280 + }, + { + "budget": 0.22, + "threshold": 1.4575860500335693, + "operator": "ge", + "actual_fraction": 0.20439414114513982, + "metric": 0.10650531263448818, + "rescued": 1067, + "harmed": 238, + "missed_rescue": 2247, + "avoided_harm": 1707, + "selected_both_correct": 857, + "selected_both_wrong": 294 + }, + { + "budget": 0.23, + "threshold": 1.4396969079971313, + "operator": "ge", + "actual_fraction": 0.2097203728362184, + "metric": 0.10689937075786074, + "rescued": 1098, + "harmed": 243, + "missed_rescue": 2216, + "avoided_harm": 1702, + "selected_both_correct": 881, + "selected_both_wrong": 298 + }, + { + "budget": 0.24, + "threshold": 1.389216661453247, + "operator": "ge", + "actual_fraction": 0.22303595206391477, + "metric": 0.10923474774365519, + "rescued": 1171, + "harmed": 265, + "missed_rescue": 2143, + "avoided_harm": 1680, + "selected_both_correct": 929, + "selected_both_wrong": 315 + }, + { + "budget": 0.25, + "threshold": 1.34909188747406, + "operator": "ge", + "actual_fraction": 0.2393475366178429, + "metric": 0.11076315105713906, + "rescued": 1264, + "harmed": 301, + "missed_rescue": 2050, + "avoided_harm": 1644, + "selected_both_correct": 982, + "selected_both_wrong": 329 + }, + { + "budget": 0.26, + "threshold": 1.3260160684585571, + "operator": "ge", + "actual_fraction": 0.24866844207723035, + "metric": 0.11386415453468966, + "rescued": 1314, + "harmed": 319, + "missed_rescue": 2000, + "avoided_harm": 1626, + "selected_both_correct": 1013, + "selected_both_wrong": 342 + }, + { + "budget": 0.27, + "threshold": 1.2911567687988281, + "operator": "ge", + "actual_fraction": 0.25998668442077233, + "metric": 0.11525658092186075, + "rescued": 1360, + "harmed": 332, + "missed_rescue": 1954, + "avoided_harm": 1613, + "selected_both_correct": 1068, + "selected_both_wrong": 364 + }, + { + "budget": 0.28, + "threshold": 1.269096851348877, + "operator": "ge", + "actual_fraction": 0.2673102529960053, + "metric": 0.11742829906658964, + "rescued": 1404, + "harmed": 342, + "missed_rescue": 1910, + "avoided_harm": 1603, + "selected_both_correct": 1099, + "selected_both_wrong": 367 + }, + { + "budget": 0.29, + "threshold": 1.2518491744995117, + "operator": "ge", + "actual_fraction": 0.2729693741677763, + "metric": 0.11870670615479877, + "rescued": 1435, + "harmed": 353, + "missed_rescue": 1879, + "avoided_harm": 1592, + "selected_both_correct": 1120, + "selected_both_wrong": 372 + }, + { + "budget": 0.3, + "threshold": 1.2406281232833862, + "operator": "ge", + "actual_fraction": 0.27862849533954726, + "metric": 0.12152711680922833, + "rescued": 1463, + "harmed": 363, + "missed_rescue": 1851, + "avoided_harm": 1582, + "selected_both_correct": 1142, + "selected_both_wrong": 380 + }, + { + "budget": 0.31, + "threshold": 1.2157225608825684, + "operator": "ge", + "actual_fraction": 0.28395472703062585, + "metric": 0.12170206188522424, + "rescued": 1494, + "harmed": 374, + "missed_rescue": 1820, + "avoided_harm": 1571, + "selected_both_correct": 1161, + "selected_both_wrong": 383 + }, + { + "budget": 0.32, + "threshold": 1.2011241912841797, + "operator": "ge", + "actual_fraction": 0.2872836218375499, + "metric": 0.12250073909335449, + "rescued": 1511, + "harmed": 383, + "missed_rescue": 1803, + "avoided_harm": 1562, + "selected_both_correct": 1171, + "selected_both_wrong": 387 + }, + { + "budget": 0.33, + "threshold": 1.1690080165863037, + "operator": "ge", + "actual_fraction": 0.29527296937416775, + "metric": 0.12431659946284873, + "rescued": 1560, + "harmed": 396, + "missed_rescue": 1754, + "avoided_harm": 1549, + "selected_both_correct": 1197, + "selected_both_wrong": 395 + }, + { + "budget": 0.34, + "threshold": 1.1319067478179932, + "operator": "ge", + "actual_fraction": 0.30892143808255657, + "metric": 0.12380518315900207, + "rescued": 1613, + "harmed": 418, + "missed_rescue": 1701, + "avoided_harm": 1527, + "selected_both_correct": 1266, + "selected_both_wrong": 415 + }, + { + "budget": 0.35, + "threshold": 1.0817104578018188, + "operator": "ge", + "actual_fraction": 0.3258988015978695, + "metric": 0.12524258698743398, + "rescued": 1688, + "harmed": 453, + "missed_rescue": 1626, + "avoided_harm": 1492, + "selected_both_correct": 1341, + "selected_both_wrong": 434 + }, + { + "budget": 0.36, + "threshold": 1.050752878189087, + "operator": "ge", + "actual_fraction": 0.33788282290279625, + "metric": 0.12739203969597912, + "rescued": 1742, + "harmed": 474, + "missed_rescue": 1572, + "avoided_harm": 1471, + "selected_both_correct": 1398, + "selected_both_wrong": 446 + }, + { + "budget": 0.37, + "threshold": 1.0408045053482056, + "operator": "ge", + "actual_fraction": 0.34354194407456723, + "metric": 0.1266684662713274, + "rescued": 1767, + "harmed": 486, + "missed_rescue": 1547, + "avoided_harm": 1459, + "selected_both_correct": 1423, + "selected_both_wrong": 452 + }, + { + "budget": 0.38, + "threshold": 1.0130165815353394, + "operator": "ge", + "actual_fraction": 0.35386151797603194, + "metric": 0.12946955595828621, + "rescued": 1811, + "harmed": 498, + "missed_rescue": 1503, + "avoided_harm": 1447, + "selected_both_correct": 1479, + "selected_both_wrong": 464 + }, + { + "budget": 0.39, + "threshold": 0.9877941012382507, + "operator": "ge", + "actual_fraction": 0.36684420772303594, + "metric": 0.13200417951747406, + "rescued": 1870, + "harmed": 524, + "missed_rescue": 1444, + "avoided_harm": 1421, + "selected_both_correct": 1539, + "selected_both_wrong": 475 + }, + { + "budget": 0.4, + "threshold": 0.9319636821746826, + "operator": "ge", + "actual_fraction": 0.39081225033288947, + "metric": 0.13488618817461398, + "rescued": 1971, + "harmed": 566, + "missed_rescue": 1343, + "avoided_harm": 1379, + "selected_both_correct": 1659, + "selected_both_wrong": 500 + }, + { + "budget": 0.41, + "threshold": 0.8980667591094971, + "operator": "ge", + "actual_fraction": 0.40346205059920104, + "metric": 0.13643326568563097, + "rescued": 2030, + "harmed": 594, + "missed_rescue": 1284, + "avoided_harm": 1351, + "selected_both_correct": 1718, + "selected_both_wrong": 506 + }, + { + "budget": 0.42, + "threshold": 0.8735002279281616, + "operator": "ge", + "actual_fraction": 0.4151131824234354, + "metric": 0.13805581747693704, + "rescued": 2080, + "harmed": 611, + "missed_rescue": 1234, + "avoided_harm": 1334, + "selected_both_correct": 1784, + "selected_both_wrong": 513 + }, + { + "budget": 0.43, + "threshold": 0.8648159503936768, + "operator": "ge", + "actual_fraction": 0.4194407456724368, + "metric": 0.13856552432216815, + "rescued": 2102, + "harmed": 620, + "missed_rescue": 1212, + "avoided_harm": 1325, + "selected_both_correct": 1804, + "selected_both_wrong": 514 + }, + { + "budget": 0.44, + "threshold": 0.8149229288101196, + "operator": "ge", + "actual_fraction": 0.4394141145139814, + "metric": 0.14002692892765436, + "rescued": 2193, + "harmed": 664, + "missed_rescue": 1121, + "avoided_harm": 1281, + "selected_both_correct": 1894, + "selected_both_wrong": 529 + }, + { + "budget": 0.45, + "threshold": 0.7909369468688965, + "operator": "ge", + "actual_fraction": 0.4517310252996005, + "metric": 0.14218965785726903, + "rescued": 2252, + "harmed": 694, + "missed_rescue": 1062, + "avoided_harm": 1251, + "selected_both_correct": 1945, + "selected_both_wrong": 537 + }, + { + "budget": 0.46, + "threshold": 0.7727278470993042, + "operator": "ge", + "actual_fraction": 0.46171770972037285, + "metric": 0.1433299730159918, + "rescued": 2298, + "harmed": 713, + "missed_rescue": 1016, + "avoided_harm": 1232, + "selected_both_correct": 1996, + "selected_both_wrong": 541 + }, + { + "budget": 0.47, + "threshold": 0.7434355020523071, + "operator": "ge", + "actual_fraction": 0.47902796271637815, + "metric": 0.14548468807591688, + "rescued": 2371, + "harmed": 754, + "missed_rescue": 943, + "avoided_harm": 1191, + "selected_both_correct": 2073, + "selected_both_wrong": 558 + }, + { + "budget": 0.48, + "threshold": 0.7266849279403687, + "operator": "ge", + "actual_fraction": 0.49134487350199735, + "metric": 0.14901560857443213, + "rescued": 2425, + "harmed": 780, + "missed_rescue": 889, + "avoided_harm": 1165, + "selected_both_correct": 2128, + "selected_both_wrong": 571 + }, + { + "budget": 0.49, + "threshold": 0.7074867486953735, + "operator": "ge", + "actual_fraction": 0.5003328894806924, + "metric": 0.14995504586743008, + "rescued": 2462, + "harmed": 802, + "missed_rescue": 852, + "avoided_harm": 1143, + "selected_both_correct": 2174, + "selected_both_wrong": 574 + }, + { + "budget": 0.5, + "threshold": 0.6545048356056213, + "operator": "ge", + "actual_fraction": 0.521970705725699, + "metric": 0.15558956072516103, + "rescued": 2558, + "harmed": 848, + "missed_rescue": 756, + "avoided_harm": 1097, + "selected_both_correct": 2275, + "selected_both_wrong": 591 + }, + { + "budget": 0.51, + "threshold": 0.6124920845031738, + "operator": "ge", + "actual_fraction": 0.5362849533954727, + "metric": 0.16090977353827407, + "rescued": 2616, + "harmed": 879, + "missed_rescue": 698, + "avoided_harm": 1066, + "selected_both_correct": 2347, + "selected_both_wrong": 602 + }, + { + "budget": 0.52, + "threshold": 0.5968987941741943, + "operator": "ge", + "actual_fraction": 0.5429427430093209, + "metric": 0.16250403847435593, + "rescued": 2642, + "harmed": 892, + "missed_rescue": 672, + "avoided_harm": 1053, + "selected_both_correct": 2382, + "selected_both_wrong": 608 + }, + { + "budget": 0.53, + "threshold": 0.572314977645874, + "operator": "ge", + "actual_fraction": 0.5496005326231691, + "metric": 0.1635851246185966, + "rescued": 2664, + "harmed": 905, + "missed_rescue": 650, + "avoided_harm": 1040, + "selected_both_correct": 2420, + "selected_both_wrong": 615 + }, + { + "budget": 0.54, + "threshold": 0.5496032238006592, + "operator": "ge", + "actual_fraction": 0.5569241011984021, + "metric": 0.1649305682067861, + "rescued": 2690, + "harmed": 923, + "missed_rescue": 624, + "avoided_harm": 1022, + "selected_both_correct": 2462, + "selected_both_wrong": 617 + }, + { + "budget": 0.55, + "threshold": 0.5177246928215027, + "operator": "ge", + "actual_fraction": 0.5629161118508655, + "metric": 0.16751708044890729, + "rescued": 2708, + "harmed": 933, + "missed_rescue": 606, + "avoided_harm": 1012, + "selected_both_correct": 2504, + "selected_both_wrong": 619 + }, + { + "budget": 0.56, + "threshold": 0.4967881739139557, + "operator": "ge", + "actual_fraction": 0.5695739014647138, + "metric": 0.17156841416816399, + "rescued": 2729, + "harmed": 944, + "missed_rescue": 585, + "avoided_harm": 1001, + "selected_both_correct": 2551, + "selected_both_wrong": 620 + }, + { + "budget": 0.57, + "threshold": 0.4586660861968994, + "operator": "ge", + "actual_fraction": 0.5805592543275633, + "metric": 0.1733981020684455, + "rescued": 2763, + "harmed": 972, + "missed_rescue": 551, + "avoided_harm": 973, + "selected_both_correct": 2619, + "selected_both_wrong": 622 + }, + { + "budget": 0.58, + "threshold": 0.4310711920261383, + "operator": "ge", + "actual_fraction": 0.5888814913448736, + "metric": 0.17507742437630908, + "rescued": 2785, + "harmed": 987, + "missed_rescue": 529, + "avoided_harm": 958, + "selected_both_correct": 2680, + "selected_both_wrong": 624 + }, + { + "budget": 0.59, + "threshold": 0.39361393451690674, + "operator": "ge", + "actual_fraction": 0.5962050599201065, + "metric": 0.17759047306833103, + "rescued": 2799, + "harmed": 1003, + "missed_rescue": 515, + "avoided_harm": 942, + "selected_both_correct": 2729, + "selected_both_wrong": 633 + }, + { + "budget": 0.6, + "threshold": 0.3438398838043213, + "operator": "ge", + "actual_fraction": 0.6078561917443409, + "metric": 0.18018202615840195, + "rescued": 2834, + "harmed": 1028, + "missed_rescue": 480, + "avoided_harm": 917, + "selected_both_correct": 2804, + "selected_both_wrong": 638 + }, + { + "budget": 0.61, + "threshold": 0.31072938442230225, + "operator": "ge", + "actual_fraction": 0.6178428761651131, + "metric": 0.1802506566427193, + "rescued": 2860, + "harmed": 1050, + "missed_rescue": 454, + "avoided_harm": 895, + "selected_both_correct": 2871, + "selected_both_wrong": 643 + }, + { + "budget": 0.62, + "threshold": 0.3001859486103058, + "operator": "ge", + "actual_fraction": 0.62017310252996, + "metric": 0.180760889750839, + "rescued": 2867, + "harmed": 1054, + "missed_rescue": 447, + "avoided_harm": 891, + "selected_both_correct": 2888, + "selected_both_wrong": 643 + }, + { + "budget": 0.63, + "threshold": 0.2700633108615875, + "operator": "ge", + "actual_fraction": 0.6324900133155792, + "metric": 0.18376666084534993, + "rescued": 2900, + "harmed": 1081, + "missed_rescue": 414, + "avoided_harm": 864, + "selected_both_correct": 2973, + "selected_both_wrong": 646 + }, + { + "budget": 0.64, + "threshold": 0.24940329790115356, + "operator": "ge", + "actual_fraction": 0.6401464713715047, + "metric": 0.18487229867011626, + "rescued": 2919, + "harmed": 1101, + "missed_rescue": 395, + "avoided_harm": 844, + "selected_both_correct": 3024, + "selected_both_wrong": 648 + }, + { + "budget": 0.65, + "threshold": 0.21490943431854248, + "operator": "ge", + "actual_fraction": 0.6554593874833555, + "metric": 0.1891377341476743, + "rescued": 2964, + "harmed": 1139, + "missed_rescue": 350, + "avoided_harm": 806, + "selected_both_correct": 3122, + "selected_both_wrong": 651 + }, + { + "budget": 0.66, + "threshold": 0.18740108609199524, + "operator": "ge", + "actual_fraction": 0.6697736351531292, + "metric": 0.19137695198725818, + "rescued": 2999, + "harmed": 1170, + "missed_rescue": 315, + "avoided_harm": 775, + "selected_both_correct": 3222, + "selected_both_wrong": 657 + }, + { + "budget": 0.67, + "threshold": 0.17519652843475342, + "operator": "ge", + "actual_fraction": 0.6744340878828229, + "metric": 0.19246697754867126, + "rescued": 3011, + "harmed": 1179, + "missed_rescue": 303, + "avoided_harm": 766, + "selected_both_correct": 3256, + "selected_both_wrong": 658 + }, + { + "budget": 0.68, + "threshold": 0.13710853457450867, + "operator": "ge", + "actual_fraction": 0.6870838881491345, + "metric": 0.19711863253876152, + "rescued": 3047, + "harmed": 1204, + "missed_rescue": 267, + "avoided_harm": 741, + "selected_both_correct": 3344, + "selected_both_wrong": 661 + }, + { + "budget": 0.69, + "threshold": 0.11163881421089172, + "operator": "ge", + "actual_fraction": 0.6964047936085219, + "metric": 0.19911481226842961, + "rescued": 3069, + "harmed": 1231, + "missed_rescue": 245, + "avoided_harm": 714, + "selected_both_correct": 3403, + "selected_both_wrong": 665 + }, + { + "budget": 0.7, + "threshold": 0.0702638104557991, + "operator": "ge", + "actual_fraction": 0.7117177097203728, + "metric": 0.20223016780448588, + "rescued": 3110, + "harmed": 1269, + "missed_rescue": 204, + "avoided_harm": 676, + "selected_both_correct": 3506, + "selected_both_wrong": 667 + }, + { + "budget": 0.71, + "threshold": 0.034758418798446655, + "operator": "ge", + "actual_fraction": 0.7213715046604527, + "metric": 0.20500205030893776, + "rescued": 3135, + "harmed": 1293, + "missed_rescue": 179, + "avoided_harm": 652, + "selected_both_correct": 3569, + "selected_both_wrong": 671 + }, + { + "budget": 0.72, + "threshold": 0.01772361993789673, + "operator": "ge", + "actual_fraction": 0.7266977363515313, + "metric": 0.20659921784927138, + "rescued": 3150, + "harmed": 1307, + "missed_rescue": 164, + "avoided_harm": 638, + "selected_both_correct": 3603, + "selected_both_wrong": 672 + }, + { + "budget": 0.73, + "threshold": -0.061358511447906494, + "operator": "ge", + "actual_fraction": 0.7490013315579228, + "metric": 0.21607081681269946, + "rescued": 3208, + "harmed": 1366, + "missed_rescue": 106, + "avoided_harm": 579, + "selected_both_correct": 3745, + "selected_both_wrong": 681 + }, + { + "budget": 0.74, + "threshold": -0.10378703474998474, + "operator": "ge", + "actual_fraction": 0.7626498002663116, + "metric": 0.2207744928635374, + "rescued": 3243, + "harmed": 1403, + "missed_rescue": 71, + "avoided_harm": 542, + "selected_both_correct": 3834, + "selected_both_wrong": 684 + }, + { + "budget": 0.75, + "threshold": -0.1483374536037445, + "operator": "ge", + "actual_fraction": 0.770306258322237, + "metric": 0.22546573365591274, + "rescued": 3265, + "harmed": 1422, + "missed_rescue": 49, + "avoided_harm": 523, + "selected_both_correct": 3884, + "selected_both_wrong": 685 + }, + { + "budget": 0.76, + "threshold": -0.20967978239059448, + "operator": "ge", + "actual_fraction": 0.7749667110519307, + "metric": 0.22848290103741348, + "rescued": 3278, + "harmed": 1438, + "missed_rescue": 36, + "avoided_harm": 507, + "selected_both_correct": 3910, + "selected_both_wrong": 686 + }, + { + "budget": 0.77, + "threshold": -0.3404620289802551, + "operator": "ge", + "actual_fraction": 0.7836218375499334, + "metric": 0.23284682057764852, + "rescued": 3300, + "harmed": 1458, + "missed_rescue": 14, + "avoided_harm": 487, + "selected_both_correct": 3968, + "selected_both_wrong": 690 + }, + { + "budget": 0.78, + "threshold": -2.93597412109375, + "operator": "ge", + "actual_fraction": 0.7882822902796272, + "metric": 0.22996109134220227, + "rescued": 3304, + "harmed": 1470, + "missed_rescue": 10, + "avoided_harm": 475, + "selected_both_correct": 4007, + "selected_both_wrong": 691 + }, + { + "budget": 0.79, + "threshold": -3.1307332515716553, + "operator": "ge", + "actual_fraction": 0.7956058588548602, + "metric": 0.22987136582739676, + "rescued": 3305, + "harmed": 1488, + "missed_rescue": 9, + "avoided_harm": 457, + "selected_both_correct": 4076, + "selected_both_wrong": 691 + }, + { + "budget": 0.8, + "threshold": -3.253781795501709, + "operator": "ge", + "actual_fraction": 0.8049267643142477, + "metric": 0.22916894688752282, + "rescued": 3306, + "harmed": 1515, + "missed_rescue": 8, + "avoided_harm": 430, + "selected_both_correct": 4160, + "selected_both_wrong": 691 + }, + { + "budget": 0.81, + "threshold": -3.405195713043213, + "operator": "ge", + "actual_fraction": 0.8192410119840213, + "metric": 0.22722094776607962, + "rescued": 3306, + "harmed": 1554, + "missed_rescue": 8, + "avoided_harm": 391, + "selected_both_correct": 4292, + "selected_both_wrong": 692 + }, + { + "budget": 0.82, + "threshold": -3.473080635070801, + "operator": "ge", + "actual_fraction": 0.8268974700399467, + "metric": 0.22672722604706697, + "rescued": 3306, + "harmed": 1571, + "missed_rescue": 8, + "avoided_harm": 374, + "selected_both_correct": 4365, + "selected_both_wrong": 694 + }, + { + "budget": 0.83, + "threshold": -3.546388864517212, + "operator": "ge", + "actual_fraction": 0.8368841544607191, + "metric": 0.2265226662772391, + "rescued": 3307, + "harmed": 1594, + "missed_rescue": 7, + "avoided_harm": 351, + "selected_both_correct": 4461, + "selected_both_wrong": 694 + }, + { + "budget": 0.84, + "threshold": -3.6391501426696777, + "operator": "ge", + "actual_fraction": 0.8498668442077231, + "metric": 0.22601721583938061, + "rescued": 3308, + "harmed": 1623, + "missed_rescue": 6, + "avoided_harm": 322, + "selected_both_correct": 4586, + "selected_both_wrong": 695 + }, + { + "budget": 0.85, + "threshold": -3.692823886871338, + "operator": "ge", + "actual_fraction": 0.8558588548601864, + "metric": 0.22576881413584862, + "rescued": 3308, + "harmed": 1636, + "missed_rescue": 6, + "avoided_harm": 309, + "selected_both_correct": 4645, + "selected_both_wrong": 695 + }, + { + "budget": 0.86, + "threshold": -3.7845706939697266, + "operator": "ge", + "actual_fraction": 0.8688415446071904, + "metric": 0.22547585678671636, + "rescued": 3309, + "harmed": 1668, + "missed_rescue": 5, + "avoided_harm": 277, + "selected_both_correct": 4768, + "selected_both_wrong": 695 + }, + { + "budget": 0.87, + "threshold": -3.833963394165039, + "operator": "ge", + "actual_fraction": 0.8761651131824234, + "metric": 0.2255463087170728, + "rescued": 3310, + "harmed": 1684, + "missed_rescue": 4, + "avoided_harm": 261, + "selected_both_correct": 4839, + "selected_both_wrong": 695 + }, + { + "budget": 0.88, + "threshold": -3.90220308303833, + "operator": "ge", + "actual_fraction": 0.8864846870838882, + "metric": 0.22534626887840087, + "rescued": 3311, + "harmed": 1706, + "missed_rescue": 3, + "avoided_harm": 239, + "selected_both_correct": 4939, + "selected_both_wrong": 696 + }, + { + "budget": 0.89, + "threshold": -3.9384937286376953, + "operator": "ge", + "actual_fraction": 0.8921438082556591, + "metric": 0.2286007208484198, + "rescued": 3312, + "harmed": 1718, + "missed_rescue": 2, + "avoided_harm": 227, + "selected_both_correct": 4994, + "selected_both_wrong": 696 + }, + { + "budget": 0.9, + "threshold": -3.957301616668701, + "operator": "ge", + "actual_fraction": 0.8954727030625832, + "metric": 0.22853275200266024, + "rescued": 3312, + "harmed": 1722, + "missed_rescue": 2, + "avoided_harm": 223, + "selected_both_correct": 5030, + "selected_both_wrong": 696 + }, + { + "budget": 0.91, + "threshold": -4.0616135597229, + "operator": "ge", + "actual_fraction": 0.914780292942743, + "metric": 0.22695603852698265, + "rescued": 3312, + "harmed": 1768, + "missed_rescue": 2, + "avoided_harm": 177, + "selected_both_correct": 5215, + "selected_both_wrong": 697 + }, + { + "budget": 0.92, + "threshold": -4.146303653717041, + "operator": "ge", + "actual_fraction": 0.9280958721704394, + "metric": 0.22602203262650622, + "rescued": 3312, + "harmed": 1797, + "missed_rescue": 2, + "avoided_harm": 148, + "selected_both_correct": 5346, + "selected_both_wrong": 697 + }, + { + "budget": 0.93, + "threshold": -4.184908390045166, + "operator": "ge", + "actual_fraction": 0.9314247669773635, + "metric": 0.2258855588897701, + "rescued": 3312, + "harmed": 1806, + "missed_rescue": 2, + "avoided_harm": 139, + "selected_both_correct": 5377, + "selected_both_wrong": 697 + }, + { + "budget": 0.94, + "threshold": -4.23890495300293, + "operator": "ge", + "actual_fraction": 0.9374167776298269, + "metric": 0.22557582859480957, + "rescued": 3312, + "harmed": 1816, + "missed_rescue": 2, + "avoided_harm": 129, + "selected_both_correct": 5439, + "selected_both_wrong": 697 + }, + { + "budget": 0.95, + "threshold": -4.278663635253906, + "operator": "ge", + "actual_fraction": 0.9417443408788282, + "metric": 0.2254347373256534, + "rescued": 3312, + "harmed": 1825, + "missed_rescue": 2, + "avoided_harm": 120, + "selected_both_correct": 5482, + "selected_both_wrong": 697 + }, + { + "budget": 0.96, + "threshold": -4.356467247009277, + "operator": "ge", + "actual_fraction": 0.9494007989347537, + "metric": 0.2251162752280064, + "rescued": 3312, + "harmed": 1846, + "missed_rescue": 2, + "avoided_harm": 99, + "selected_both_correct": 5553, + "selected_both_wrong": 697 + }, + { + "budget": 0.97, + "threshold": -4.431670188903809, + "operator": "ge", + "actual_fraction": 0.9573901464713716, + "metric": 0.22526637897204788, + "rescued": 3313, + "harmed": 1861, + "missed_rescue": 1, + "avoided_harm": 84, + "selected_both_correct": 5633, + "selected_both_wrong": 697 + }, + { + "budget": 0.98, + "threshold": -4.527087688446045, + "operator": "ge", + "actual_fraction": 0.9670439414114514, + "metric": 0.22501890928122348, + "rescued": 3313, + "harmed": 1878, + "missed_rescue": 1, + "avoided_harm": 67, + "selected_both_correct": 5732, + "selected_both_wrong": 697 + }, + { + "budget": 0.99, + "threshold": -4.6759443283081055, + "operator": "ge", + "actual_fraction": 0.9790279627163782, + "metric": 0.22374120573345554, + "rescued": 3313, + "harmed": 1903, + "missed_rescue": 1, + "avoided_harm": 42, + "selected_both_correct": 5851, + "selected_both_wrong": 697 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.22361115727328065, + "rescued": 3314, + "harmed": 1945, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 6060, + "selected_both_wrong": 697 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 1391, + "rescuable": 786, + "harmful": 475, + "both_wrong": 152, + "unit": "event", + "total": 2804 + }, + "test": { + "both_correct": 6060, + "rescuable": 3314, + "harmful": 1945, + "both_wrong": 697, + "unit": "event", + "total": 12016 + } + }, + "small": { + "validation": 0.07641657905799261, + "test": 0.07887397871043929 + }, + "large": { + "validation": 0.20666016863629594, + "test": 0.22361115727328065 + }, + "learned_minus_confidence_test": 0.013865962618061306, + "learned_minus_small_test": 0.151087112631763, + "learned_minus_large_test": 0.00634993406892162 + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "Qwen3.8" + ], + "expert_display": "RT-DETR-X", + "run": "shared_12_rtdetr_x_fp32_Qwen3.8", + "validation_rows": 701, + "test_rows": 3004, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detection_count", + "has_detections", + "max_score", + "mean_score", + "detection_count_valid", + "has_detections_valid", + "max_score_valid", + "mean_score_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/rtdetr_x_fp32/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 99, + "validation_area_0_25": 0.10053453664183931, + "validation_point": { + "budget": 0.99, + "threshold": -0.44162338972091675, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.24862274646934207, + "rescued": 805, + "harmed": 184, + "missed_rescue": 10, + "avoided_harm": 1, + "selected_both_correct": 1662, + "selected_both_wrong": 121 + }, + "test_point": { + "budget": 0.99, + "threshold": -0.44162338972091675, + "operator": "ge", + "actual_fraction": 0.9860186418109188, + "metric": 0.26818431283513144, + "rescued": 3404, + "harmed": 671, + "missed_rescue": 57, + "avoided_harm": 5, + "selected_both_correct": 7228, + "selected_both_wrong": 545 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.07641657905799261, + "rescued": 0, + "harmed": 0, + "missed_rescue": 815, + "avoided_harm": 185, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.021043073385953903, + "operator": "ge", + "actual_fraction": 0.009985734664764621, + "metric": 0.07641657905799261, + "rescued": 0, + "harmed": 1, + "missed_rescue": 815, + "avoided_harm": 184, + "selected_both_correct": 27, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -0.022327054291963577, + "operator": "ge", + "actual_fraction": 0.019971469329529243, + "metric": 0.07641657905799261, + "rescued": 0, + "harmed": 2, + "missed_rescue": 815, + "avoided_harm": 183, + "selected_both_correct": 54, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -0.024909960106015205, + "operator": "ge", + "actual_fraction": 0.029957203994293864, + "metric": 0.07641657905799261, + "rescued": 0, + "harmed": 5, + "missed_rescue": 815, + "avoided_harm": 180, + "selected_both_correct": 79, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": -0.027814723551273346, + "operator": "ge", + "actual_fraction": 0.039942938659058486, + "metric": 0.07641657905799261, + "rescued": 0, + "harmed": 5, + "missed_rescue": 815, + "avoided_harm": 180, + "selected_both_correct": 107, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": -0.028914429247379303, + "operator": "ge", + "actual_fraction": 0.04992867332382311, + "metric": 0.10273236853167682, + "rescued": 1, + "harmed": 5, + "missed_rescue": 814, + "avoided_harm": 180, + "selected_both_correct": 134, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": -0.030170045793056488, + "operator": "ge", + "actual_fraction": 0.05991440798858773, + "metric": 0.10349184930294494, + "rescued": 2, + "harmed": 6, + "missed_rescue": 813, + "avoided_harm": 179, + "selected_both_correct": 160, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": -0.03123854100704193, + "operator": "ge", + "actual_fraction": 0.06990014265335236, + "metric": 0.10193024779448791, + "rescued": 2, + "harmed": 10, + "missed_rescue": 813, + "avoided_harm": 175, + "selected_both_correct": 184, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": -0.03410686179995537, + "operator": "ge", + "actual_fraction": 0.07988587731811697, + "metric": 0.10070576146899149, + "rescued": 2, + "harmed": 12, + "missed_rescue": 813, + "avoided_harm": 173, + "selected_both_correct": 210, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": -0.03476816788315773, + "operator": "ge", + "actual_fraction": 0.0898716119828816, + "metric": 0.09863536809425029, + "rescued": 2, + "harmed": 14, + "missed_rescue": 813, + "avoided_harm": 171, + "selected_both_correct": 236, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": -0.0365506075322628, + "operator": "ge", + "actual_fraction": 0.09985734664764621, + "metric": 0.09772957099280101, + "rescued": 2, + "harmed": 15, + "missed_rescue": 813, + "avoided_harm": 170, + "selected_both_correct": 263, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": -0.03818900138139725, + "operator": "ge", + "actual_fraction": 0.10984308131241084, + "metric": 0.09678681181213102, + "rescued": 2, + "harmed": 17, + "missed_rescue": 813, + "avoided_harm": 168, + "selected_both_correct": 289, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": -0.039673544466495514, + "operator": "ge", + "actual_fraction": 0.11982881597717546, + "metric": 0.0951962932074597, + "rescued": 2, + "harmed": 20, + "missed_rescue": 813, + "avoided_harm": 165, + "selected_both_correct": 314, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": -0.04085410386323929, + "operator": "ge", + "actual_fraction": 0.12981455064194009, + "metric": 0.09473838918283363, + "rescued": 2, + "harmed": 25, + "missed_rescue": 813, + "avoided_harm": 160, + "selected_both_correct": 337, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": -0.04204101860523224, + "operator": "ge", + "actual_fraction": 0.1398002853067047, + "metric": 0.09404328215803626, + "rescued": 2, + "harmed": 27, + "missed_rescue": 813, + "avoided_harm": 158, + "selected_both_correct": 363, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": -0.04278091713786125, + "operator": "ge", + "actual_fraction": 0.14978601997146934, + "metric": 0.10930560071560966, + "rescued": 3, + "harmed": 30, + "missed_rescue": 812, + "avoided_harm": 155, + "selected_both_correct": 386, + "selected_both_wrong": 1 + }, + { + "budget": 0.16, + "threshold": -0.04322759434580803, + "operator": "ge", + "actual_fraction": 0.15977175463623394, + "metric": 0.10819687589305238, + "rescued": 3, + "harmed": 32, + "missed_rescue": 812, + "avoided_harm": 153, + "selected_both_correct": 412, + "selected_both_wrong": 1 + }, + { + "budget": 0.17, + "threshold": -0.04394488036632538, + "operator": "ge", + "actual_fraction": 0.16975748930099857, + "metric": 0.10718881137692335, + "rescued": 3, + "harmed": 33, + "missed_rescue": 812, + "avoided_harm": 152, + "selected_both_correct": 439, + "selected_both_wrong": 1 + }, + { + "budget": 0.18, + "threshold": -0.04530126228928566, + "operator": "ge", + "actual_fraction": 0.1797432239657632, + "metric": 0.106004048568, + "rescued": 3, + "harmed": 37, + "missed_rescue": 812, + "avoided_harm": 148, + "selected_both_correct": 463, + "selected_both_wrong": 1 + }, + { + "budget": 0.19, + "threshold": -0.04661082476377487, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.106004048568, + "rescued": 3, + "harmed": 37, + "missed_rescue": 812, + "avoided_harm": 148, + "selected_both_correct": 491, + "selected_both_wrong": 1 + }, + { + "budget": 0.2, + "threshold": -0.04730507358908653, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.10416681691465626, + "rescued": 3, + "harmed": 40, + "missed_rescue": 812, + "avoided_harm": 145, + "selected_both_correct": 516, + "selected_both_wrong": 1 + }, + { + "budget": 0.21, + "threshold": -0.048246145248413086, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.10337316612100547, + "rescued": 3, + "harmed": 41, + "missed_rescue": 812, + "avoided_harm": 144, + "selected_both_correct": 543, + "selected_both_wrong": 1 + }, + { + "budget": 0.22, + "threshold": -0.04900278151035309, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.10251715413933904, + "rescued": 3, + "harmed": 43, + "missed_rescue": 812, + "avoided_harm": 142, + "selected_both_correct": 569, + "selected_both_wrong": 1 + }, + { + "budget": 0.23, + "threshold": -0.050042688846588135, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.10180591658599622, + "rescued": 3, + "harmed": 44, + "missed_rescue": 812, + "avoided_harm": 141, + "selected_both_correct": 595, + "selected_both_wrong": 2 + }, + { + "budget": 0.24, + "threshold": -0.050901688635349274, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.10179370893656609, + "rescued": 9, + "harmed": 48, + "missed_rescue": 806, + "avoided_harm": 137, + "selected_both_correct": 612, + "selected_both_wrong": 3 + }, + { + "budget": 0.25, + "threshold": -0.05161922052502632, + "operator": "ge", + "actual_fraction": 0.24964336661911554, + "metric": 0.10106007305091304, + "rescued": 16, + "harmed": 50, + "missed_rescue": 799, + "avoided_harm": 135, + "selected_both_correct": 631, + "selected_both_wrong": 3 + }, + { + "budget": 0.26, + "threshold": -0.05245445668697357, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.10034763622704679, + "rescued": 22, + "harmed": 52, + "missed_rescue": 793, + "avoided_harm": 133, + "selected_both_correct": 650, + "selected_both_wrong": 4 + }, + { + "budget": 0.27, + "threshold": -0.053079016506671906, + "operator": "ge", + "actual_fraction": 0.26961483594864477, + "metric": 0.10038369304556356, + "rescued": 30, + "harmed": 53, + "missed_rescue": 785, + "avoided_harm": 132, + "selected_both_correct": 669, + "selected_both_wrong": 4 + }, + { + "budget": 0.28, + "threshold": -0.053604189306497574, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.10087055535678022, + "rescued": 38, + "harmed": 53, + "missed_rescue": 777, + "avoided_harm": 132, + "selected_both_correct": 686, + "selected_both_wrong": 7 + }, + { + "budget": 0.29, + "threshold": -0.05505518987774849, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.10109647230706402, + "rescued": 48, + "harmed": 54, + "missed_rescue": 767, + "avoided_harm": 131, + "selected_both_correct": 703, + "selected_both_wrong": 7 + }, + { + "budget": 0.3, + "threshold": -0.05605518817901611, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.10072202785599546, + "rescued": 54, + "harmed": 57, + "missed_rescue": 761, + "avoided_harm": 128, + "selected_both_correct": 721, + "selected_both_wrong": 8 + }, + { + "budget": 0.31, + "threshold": -0.056986160576343536, + "operator": "ge", + "actual_fraction": 0.3095577746077033, + "metric": 0.10144063504785386, + "rescued": 62, + "harmed": 60, + "missed_rescue": 753, + "avoided_harm": 125, + "selected_both_correct": 737, + "selected_both_wrong": 9 + }, + { + "budget": 0.32, + "threshold": -0.0582931749522686, + "operator": "ge", + "actual_fraction": 0.3195435092724679, + "metric": 0.1016168805568569, + "rescued": 70, + "harmed": 61, + "missed_rescue": 745, + "avoided_harm": 124, + "selected_both_correct": 756, + "selected_both_wrong": 9 + }, + { + "budget": 0.33, + "threshold": -0.0596650205552578, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.10207649449543332, + "rescued": 76, + "harmed": 61, + "missed_rescue": 739, + "avoided_harm": 124, + "selected_both_correct": 776, + "selected_both_wrong": 11 + }, + { + "budget": 0.34, + "threshold": -0.060549296438694, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.10147122890331242, + "rescued": 87, + "harmed": 65, + "missed_rescue": 728, + "avoided_harm": 120, + "selected_both_correct": 788, + "selected_both_wrong": 12 + }, + { + "budget": 0.35, + "threshold": -0.06178095191717148, + "operator": "ge", + "actual_fraction": 0.34950071326676174, + "metric": 0.11031878913558303, + "rescued": 97, + "harmed": 67, + "missed_rescue": 718, + "avoided_harm": 118, + "selected_both_correct": 803, + "selected_both_wrong": 13 + }, + { + "budget": 0.36, + "threshold": -0.06328777223825455, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.1108012937861455, + "rescued": 109, + "harmed": 68, + "missed_rescue": 706, + "avoided_harm": 117, + "selected_both_correct": 818, + "selected_both_wrong": 13 + }, + { + "budget": 0.37, + "threshold": -0.0647548958659172, + "operator": "ge", + "actual_fraction": 0.369472182596291, + "metric": 0.11116507183385692, + "rescued": 119, + "harmed": 69, + "missed_rescue": 696, + "avoided_harm": 116, + "selected_both_correct": 834, + "selected_both_wrong": 14 + }, + { + "budget": 0.38, + "threshold": -0.06637481600046158, + "operator": "ge", + "actual_fraction": 0.37945791726105566, + "metric": 0.11160958901594166, + "rescued": 128, + "harmed": 70, + "missed_rescue": 687, + "avoided_harm": 115, + "selected_both_correct": 851, + "selected_both_wrong": 15 + }, + { + "budget": 0.39, + "threshold": -0.06774259358644485, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.11234765239703698, + "rescued": 137, + "harmed": 70, + "missed_rescue": 678, + "avoided_harm": 115, + "selected_both_correct": 867, + "selected_both_wrong": 18 + }, + { + "budget": 0.4, + "threshold": -0.06843380630016327, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.11018425519727987, + "rescued": 146, + "harmed": 72, + "missed_rescue": 669, + "avoided_harm": 113, + "selected_both_correct": 883, + "selected_both_wrong": 19 + }, + { + "budget": 0.41, + "threshold": -0.0694691464304924, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.11067980295566503, + "rescued": 154, + "harmed": 76, + "missed_rescue": 661, + "avoided_harm": 109, + "selected_both_correct": 897, + "selected_both_wrong": 21 + }, + { + "budget": 0.42, + "threshold": -0.07112707942724228, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.11116873445502742, + "rescued": 166, + "harmed": 78, + "missed_rescue": 649, + "avoided_harm": 107, + "selected_both_correct": 911, + "selected_both_wrong": 21 + }, + { + "budget": 0.43, + "threshold": -0.07192998379468918, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.11166452557368921, + "rescued": 175, + "harmed": 82, + "missed_rescue": 640, + "avoided_harm": 103, + "selected_both_correct": 925, + "selected_both_wrong": 22 + }, + { + "budget": 0.44, + "threshold": -0.07266208529472351, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.11224389412088417, + "rescued": 185, + "harmed": 84, + "missed_rescue": 630, + "avoided_harm": 101, + "selected_both_correct": 940, + "selected_both_wrong": 23 + }, + { + "budget": 0.45, + "threshold": -0.0744253620505333, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.11275924414114559, + "rescued": 194, + "harmed": 85, + "missed_rescue": 621, + "avoided_harm": 100, + "selected_both_correct": 955, + "selected_both_wrong": 26 + }, + { + "budget": 0.46, + "threshold": -0.07532940804958344, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.11288099124724625, + "rescued": 203, + "harmed": 88, + "missed_rescue": 612, + "avoided_harm": 97, + "selected_both_correct": 970, + "selected_both_wrong": 27 + }, + { + "budget": 0.47, + "threshold": -0.0770750343799591, + "operator": "ge", + "actual_fraction": 0.4693295292439372, + "metric": 0.11312993997763046, + "rescued": 218, + "harmed": 91, + "missed_rescue": 597, + "avoided_harm": 94, + "selected_both_correct": 979, + "selected_both_wrong": 28 + }, + { + "budget": 0.48, + "threshold": -0.07818037271499634, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.11421587032830263, + "rescued": 227, + "harmed": 91, + "missed_rescue": 588, + "avoided_harm": 94, + "selected_both_correct": 997, + "selected_both_wrong": 29 + }, + { + "budget": 0.49, + "threshold": -0.07998209446668625, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.11231395556672395, + "rescued": 233, + "harmed": 95, + "missed_rescue": 582, + "avoided_harm": 90, + "selected_both_correct": 1014, + "selected_both_wrong": 30 + }, + { + "budget": 0.5, + "threshold": -0.08124949783086777, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.11317276185775925, + "rescued": 242, + "harmed": 95, + "missed_rescue": 573, + "avoided_harm": 90, + "selected_both_correct": 1032, + "selected_both_wrong": 31 + }, + { + "budget": 0.51, + "threshold": -0.08398465067148209, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.11397997643625071, + "rescued": 255, + "harmed": 98, + "missed_rescue": 560, + "avoided_harm": 87, + "selected_both_correct": 1044, + "selected_both_wrong": 31 + }, + { + "budget": 0.52, + "threshold": -0.08537434041500092, + "operator": "ge", + "actual_fraction": 0.5192582025677603, + "metric": 0.12321191882582982, + "rescued": 266, + "harmed": 101, + "missed_rescue": 549, + "avoided_harm": 84, + "selected_both_correct": 1057, + "selected_both_wrong": 32 + }, + { + "budget": 0.53, + "threshold": -0.08664409816265106, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.12377158600708722, + "rescued": 276, + "harmed": 103, + "missed_rescue": 539, + "avoided_harm": 82, + "selected_both_correct": 1071, + "selected_both_wrong": 34 + }, + { + "budget": 0.54, + "threshold": -0.08777308464050293, + "operator": "ge", + "actual_fraction": 0.5392296718972895, + "metric": 0.12471535828861256, + "rescued": 288, + "harmed": 105, + "missed_rescue": 527, + "avoided_harm": 80, + "selected_both_correct": 1084, + "selected_both_wrong": 35 + }, + { + "budget": 0.55, + "threshold": -0.08980843424797058, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.12627445993419065, + "rescued": 301, + "harmed": 106, + "missed_rescue": 514, + "avoided_harm": 79, + "selected_both_correct": 1098, + "selected_both_wrong": 35 + }, + { + "budget": 0.56, + "threshold": -0.0906188115477562, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.1336597715556373, + "rescued": 314, + "harmed": 107, + "missed_rescue": 501, + "avoided_harm": 78, + "selected_both_correct": 1111, + "selected_both_wrong": 36 + }, + { + "budget": 0.57, + "threshold": -0.09160211682319641, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.13507916719864918, + "rescued": 325, + "harmed": 108, + "missed_rescue": 490, + "avoided_harm": 77, + "selected_both_correct": 1126, + "selected_both_wrong": 37 + }, + { + "budget": 0.58, + "threshold": -0.09283093363046646, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.13601889857898128, + "rescued": 336, + "harmed": 110, + "missed_rescue": 479, + "avoided_harm": 75, + "selected_both_correct": 1140, + "selected_both_wrong": 38 + }, + { + "budget": 0.59, + "threshold": -0.0941123366355896, + "operator": "ge", + "actual_fraction": 0.5891583452211127, + "metric": 0.13626938313219553, + "rescued": 346, + "harmed": 114, + "missed_rescue": 469, + "avoided_harm": 71, + "selected_both_correct": 1153, + "selected_both_wrong": 39 + }, + { + "budget": 0.6, + "threshold": -0.09614869952201843, + "operator": "ge", + "actual_fraction": 0.5991440798858774, + "metric": 0.13736913676837675, + "rescued": 359, + "harmed": 116, + "missed_rescue": 456, + "avoided_harm": 69, + "selected_both_correct": 1165, + "selected_both_wrong": 40 + }, + { + "budget": 0.61, + "threshold": -0.09772664308547974, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.13874937058804088, + "rescued": 371, + "harmed": 117, + "missed_rescue": 444, + "avoided_harm": 68, + "selected_both_correct": 1177, + "selected_both_wrong": 43 + }, + { + "budget": 0.62, + "threshold": -0.09911532700061798, + "operator": "ge", + "actual_fraction": 0.6191155492154066, + "metric": 0.1401946427657629, + "rescued": 383, + "harmed": 118, + "missed_rescue": 432, + "avoided_harm": 67, + "selected_both_correct": 1190, + "selected_both_wrong": 45 + }, + { + "budget": 0.63, + "threshold": -0.10007988661527634, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.14163850172491793, + "rescued": 395, + "harmed": 119, + "missed_rescue": 420, + "avoided_harm": 66, + "selected_both_correct": 1204, + "selected_both_wrong": 46 + }, + { + "budget": 0.64, + "threshold": -0.10177135467529297, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.14631539139000474, + "rescued": 409, + "harmed": 124, + "missed_rescue": 406, + "avoided_harm": 61, + "selected_both_correct": 1210, + "selected_both_wrong": 49 + }, + { + "budget": 0.65, + "threshold": -0.10395076125860214, + "operator": "ge", + "actual_fraction": 0.6490727532097005, + "metric": 0.14751840006124584, + "rescued": 418, + "harmed": 126, + "missed_rescue": 397, + "avoided_harm": 59, + "selected_both_correct": 1226, + "selected_both_wrong": 50 + }, + { + "budget": 0.66, + "threshold": -0.10524322092533112, + "operator": "ge", + "actual_fraction": 0.6590584878744651, + "metric": 0.14849238248708915, + "rescued": 430, + "harmed": 129, + "missed_rescue": 385, + "avoided_harm": 56, + "selected_both_correct": 1236, + "selected_both_wrong": 53 + }, + { + "budget": 0.67, + "threshold": -0.10689426958560944, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.15045288951822525, + "rescued": 444, + "harmed": 130, + "missed_rescue": 371, + "avoided_harm": 55, + "selected_both_correct": 1249, + "selected_both_wrong": 53 + }, + { + "budget": 0.68, + "threshold": -0.11014843732118607, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.15143717933254464, + "rescued": 455, + "harmed": 132, + "missed_rescue": 360, + "avoided_harm": 53, + "selected_both_correct": 1261, + "selected_both_wrong": 56 + }, + { + "budget": 0.69, + "threshold": -0.11117545515298843, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.158399005658965, + "rescued": 469, + "harmed": 136, + "missed_rescue": 346, + "avoided_harm": 49, + "selected_both_correct": 1269, + "selected_both_wrong": 58 + }, + { + "budget": 0.7, + "threshold": -0.11230025440454483, + "operator": "ge", + "actual_fraction": 0.6990014265335235, + "metric": 0.15918455527939407, + "rescued": 480, + "harmed": 137, + "missed_rescue": 335, + "avoided_harm": 48, + "selected_both_correct": 1283, + "selected_both_wrong": 60 + }, + { + "budget": 0.71, + "threshold": -0.11423606425523758, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.16079826154850815, + "rescued": 491, + "harmed": 138, + "missed_rescue": 324, + "avoided_harm": 47, + "selected_both_correct": 1297, + "selected_both_wrong": 62 + }, + { + "budget": 0.72, + "threshold": -0.11647249013185501, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.16232381124917994, + "rescued": 506, + "harmed": 142, + "missed_rescue": 309, + "avoided_harm": 43, + "selected_both_correct": 1305, + "selected_both_wrong": 63 + }, + { + "budget": 0.73, + "threshold": -0.11962417513132095, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.16281139401042977, + "rescued": 516, + "harmed": 145, + "missed_rescue": 299, + "avoided_harm": 40, + "selected_both_correct": 1315, + "selected_both_wrong": 68 + }, + { + "budget": 0.74, + "threshold": -0.12126146256923676, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.16441172693350847, + "rescued": 527, + "harmed": 146, + "missed_rescue": 288, + "avoided_harm": 39, + "selected_both_correct": 1328, + "selected_both_wrong": 71 + }, + { + "budget": 0.75, + "threshold": -0.1230982318520546, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.16630001788027923, + "rescued": 538, + "harmed": 146, + "missed_rescue": 277, + "avoided_harm": 39, + "selected_both_correct": 1342, + "selected_both_wrong": 74 + }, + { + "budget": 0.76, + "threshold": -0.12517565488815308, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.16644279851003654, + "rescued": 549, + "harmed": 149, + "missed_rescue": 266, + "avoided_harm": 36, + "selected_both_correct": 1354, + "selected_both_wrong": 76 + }, + { + "budget": 0.77, + "threshold": -0.12628257274627686, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.16746586450739254, + "rescued": 556, + "harmed": 155, + "missed_rescue": 259, + "avoided_harm": 30, + "selected_both_correct": 1365, + "selected_both_wrong": 80 + }, + { + "budget": 0.78, + "threshold": -0.12975578010082245, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.1703501656136907, + "rescued": 568, + "harmed": 156, + "missed_rescue": 247, + "avoided_harm": 29, + "selected_both_correct": 1379, + "selected_both_wrong": 81 + }, + { + "budget": 0.79, + "threshold": -0.13229140639305115, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.17215569220087368, + "rescued": 577, + "harmed": 157, + "missed_rescue": 238, + "avoided_harm": 28, + "selected_both_correct": 1392, + "selected_both_wrong": 86 + }, + { + "budget": 0.8, + "threshold": -0.1373685747385025, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.1747173129779689, + "rescued": 591, + "harmed": 160, + "missed_rescue": 224, + "avoided_harm": 25, + "selected_both_correct": 1402, + "selected_both_wrong": 87 + }, + { + "budget": 0.81, + "threshold": -0.1407174915075302, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.18313694998955, + "rescued": 609, + "harmed": 160, + "missed_rescue": 206, + "avoided_harm": 25, + "selected_both_correct": 1411, + "selected_both_wrong": 88 + }, + { + "budget": 0.82, + "threshold": -0.1433146893978119, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.18621937310674674, + "rescued": 623, + "harmed": 162, + "missed_rescue": 192, + "avoided_harm": 23, + "selected_both_correct": 1421, + "selected_both_wrong": 90 + }, + { + "budget": 0.83, + "threshold": -0.1457100659608841, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.1886261591198746, + "rescued": 632, + "harmed": 163, + "missed_rescue": 183, + "avoided_harm": 22, + "selected_both_correct": 1439, + "selected_both_wrong": 90 + }, + { + "budget": 0.84, + "threshold": -0.148710235953331, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.19060328592765763, + "rescued": 642, + "harmed": 167, + "missed_rescue": 173, + "avoided_harm": 18, + "selected_both_correct": 1451, + "selected_both_wrong": 92 + }, + { + "budget": 0.85, + "threshold": -0.15291842818260193, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.1979618629476584, + "rescued": 658, + "harmed": 169, + "missed_rescue": 157, + "avoided_harm": 16, + "selected_both_correct": 1460, + "selected_both_wrong": 93 + }, + { + "budget": 0.86, + "threshold": -0.15889839828014374, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.2003153021893066, + "rescued": 667, + "harmed": 170, + "missed_rescue": 148, + "avoided_harm": 15, + "selected_both_correct": 1476, + "selected_both_wrong": 95 + }, + { + "budget": 0.87, + "threshold": -0.16629797220230103, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.20401478964488712, + "rescued": 678, + "harmed": 172, + "missed_rescue": 137, + "avoided_harm": 13, + "selected_both_correct": 1491, + "selected_both_wrong": 95 + }, + { + "budget": 0.88, + "threshold": -0.17295491695404053, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.20519541918721101, + "rescued": 687, + "harmed": 173, + "missed_rescue": 128, + "avoided_harm": 12, + "selected_both_correct": 1507, + "selected_both_wrong": 97 + }, + { + "budget": 0.89, + "threshold": -0.18053314089775085, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.20764101752605946, + "rescued": 695, + "harmed": 175, + "missed_rescue": 120, + "avoided_harm": 10, + "selected_both_correct": 1522, + "selected_both_wrong": 100 + }, + { + "budget": 0.9, + "threshold": -0.19442276656627655, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.2119684936431676, + "rescued": 709, + "harmed": 175, + "missed_rescue": 106, + "avoided_harm": 10, + "selected_both_correct": 1535, + "selected_both_wrong": 101 + }, + { + "budget": 0.91, + "threshold": -0.20900504291057587, + "operator": "ge", + "actual_fraction": 0.9087018544935807, + "metric": 0.21605348977753452, + "rescued": 719, + "harmed": 175, + "missed_rescue": 96, + "avoided_harm": 10, + "selected_both_correct": 1551, + "selected_both_wrong": 103 + }, + { + "budget": 0.92, + "threshold": -0.22383525967597961, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.2203498969356328, + "rescued": 731, + "harmed": 175, + "missed_rescue": 84, + "avoided_harm": 10, + "selected_both_correct": 1564, + "selected_both_wrong": 106 + }, + { + "budget": 0.93, + "threshold": -0.25952714681625366, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.22073473473473473, + "rescued": 741, + "harmed": 178, + "missed_rescue": 74, + "avoided_harm": 7, + "selected_both_correct": 1577, + "selected_both_wrong": 108 + }, + { + "budget": 0.94, + "threshold": -0.2804492115974426, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.21785099794688836, + "rescued": 751, + "harmed": 179, + "missed_rescue": 64, + "avoided_harm": 6, + "selected_both_correct": 1590, + "selected_both_wrong": 112 + }, + { + "budget": 0.95, + "threshold": -0.2984963357448578, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.22594140701088306, + "rescued": 764, + "harmed": 179, + "missed_rescue": 51, + "avoided_harm": 6, + "selected_both_correct": 1603, + "selected_both_wrong": 114 + }, + { + "budget": 0.96, + "threshold": -0.32223379611968994, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.23296274186615876, + "rescued": 774, + "harmed": 179, + "missed_rescue": 41, + "avoided_harm": 6, + "selected_both_correct": 1619, + "selected_both_wrong": 116 + }, + { + "budget": 0.97, + "threshold": -0.36503666639328003, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.23681731712393103, + "rescued": 784, + "harmed": 180, + "missed_rescue": 31, + "avoided_harm": 5, + "selected_both_correct": 1635, + "selected_both_wrong": 117 + }, + { + "budget": 0.98, + "threshold": -0.4221048653125763, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.24211204189132882, + "rescued": 796, + "harmed": 182, + "missed_rescue": 19, + "avoided_harm": 3, + "selected_both_correct": 1649, + "selected_both_wrong": 117 + }, + { + "budget": 0.99, + "threshold": -0.44162338972091675, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.24862274646934207, + "rescued": 805, + "harmed": 184, + "missed_rescue": 10, + "avoided_harm": 1, + "selected_both_correct": 1662, + "selected_both_wrong": 121 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.25448626222520154, + "rescued": 815, + "harmed": 185, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 1681, + "selected_both_wrong": 123 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.07887397871043929, + "rescued": 0, + "harmed": 0, + "missed_rescue": 3461, + "avoided_harm": 676, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.021043073385953903, + "operator": "ge", + "actual_fraction": 0.007989347536617843, + "metric": 0.07887397871043929, + "rescued": 0, + "harmed": 5, + "missed_rescue": 3461, + "avoided_harm": 671, + "selected_both_correct": 91, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -0.022327054291963577, + "operator": "ge", + "actual_fraction": 0.016644474034620507, + "metric": 0.07904050790892239, + "rescued": 1, + "harmed": 8, + "missed_rescue": 3460, + "avoided_harm": 668, + "selected_both_correct": 191, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -0.024909960106015205, + "operator": "ge", + "actual_fraction": 0.0296271637816245, + "metric": 0.07904050790892239, + "rescued": 1, + "harmed": 11, + "missed_rescue": 3460, + "avoided_harm": 665, + "selected_both_correct": 344, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": -0.027814723551273346, + "operator": "ge", + "actual_fraction": 0.046937416777629824, + "metric": 0.07894575740459331, + "rescued": 1, + "harmed": 19, + "missed_rescue": 3460, + "avoided_harm": 657, + "selected_both_correct": 544, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": -0.028914429247379303, + "operator": "ge", + "actual_fraction": 0.05326231691078562, + "metric": 0.07941709585555166, + "rescued": 2, + "harmed": 22, + "missed_rescue": 3459, + "avoided_harm": 654, + "selected_both_correct": 616, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": -0.030170045793056488, + "operator": "ge", + "actual_fraction": 0.05992010652463382, + "metric": 0.07940750674336808, + "rescued": 2, + "harmed": 23, + "missed_rescue": 3459, + "avoided_harm": 653, + "selected_both_correct": 695, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": -0.03123854100704193, + "operator": "ge", + "actual_fraction": 0.06724367509986684, + "metric": 0.09221541101182128, + "rescued": 5, + "harmed": 28, + "missed_rescue": 3456, + "avoided_harm": 648, + "selected_both_correct": 775, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": -0.03410686179995537, + "operator": "ge", + "actual_fraction": 0.08721704394141146, + "metric": 0.09167716576535306, + "rescued": 6, + "harmed": 39, + "missed_rescue": 3455, + "avoided_harm": 637, + "selected_both_correct": 1003, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": -0.03476816788315773, + "operator": "ge", + "actual_fraction": 0.09287616511318242, + "metric": 0.09085537954855229, + "rescued": 6, + "harmed": 46, + "missed_rescue": 3455, + "avoided_harm": 630, + "selected_both_correct": 1064, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": -0.0365506075322628, + "operator": "ge", + "actual_fraction": 0.10785619174434088, + "metric": 0.09006366890142659, + "rescued": 6, + "harmed": 56, + "missed_rescue": 3455, + "avoided_harm": 620, + "selected_both_correct": 1233, + "selected_both_wrong": 1 + }, + { + "budget": 0.11, + "threshold": -0.03818900138139725, + "operator": "ge", + "actual_fraction": 0.11784287616511319, + "metric": 0.09042296916775287, + "rescued": 8, + "harmed": 58, + "missed_rescue": 3453, + "avoided_harm": 618, + "selected_both_correct": 1349, + "selected_both_wrong": 1 + }, + { + "budget": 0.12, + "threshold": -0.039673544466495514, + "operator": "ge", + "actual_fraction": 0.12716378162450068, + "metric": 0.08993827377942153, + "rescued": 8, + "harmed": 65, + "missed_rescue": 3453, + "avoided_harm": 611, + "selected_both_correct": 1454, + "selected_both_wrong": 1 + }, + { + "budget": 0.13, + "threshold": -0.04085410386323929, + "operator": "ge", + "actual_fraction": 0.13648468708388814, + "metric": 0.08955740758160691, + "rescued": 8, + "harmed": 73, + "missed_rescue": 3453, + "avoided_harm": 603, + "selected_both_correct": 1558, + "selected_both_wrong": 1 + }, + { + "budget": 0.14, + "threshold": -0.04204101860523224, + "operator": "ge", + "actual_fraction": 0.14613848202396804, + "metric": 0.0893664971618984, + "rescued": 8, + "harmed": 76, + "missed_rescue": 3453, + "avoided_harm": 600, + "selected_both_correct": 1671, + "selected_both_wrong": 1 + }, + { + "budget": 0.15, + "threshold": -0.04278091713786125, + "operator": "ge", + "actual_fraction": 0.1514647137150466, + "metric": 0.08915747670369788, + "rescued": 8, + "harmed": 82, + "missed_rescue": 3453, + "avoided_harm": 594, + "selected_both_correct": 1729, + "selected_both_wrong": 1 + }, + { + "budget": 0.16, + "threshold": -0.04322759434580803, + "operator": "ge", + "actual_fraction": 0.15579227696404793, + "metric": 0.08910079841343506, + "rescued": 8, + "harmed": 85, + "missed_rescue": 3453, + "avoided_harm": 591, + "selected_both_correct": 1778, + "selected_both_wrong": 1 + }, + { + "budget": 0.17, + "threshold": -0.04394488036632538, + "operator": "ge", + "actual_fraction": 0.1617842876165113, + "metric": 0.08889771271713626, + "rescued": 8, + "harmed": 89, + "missed_rescue": 3453, + "avoided_harm": 587, + "selected_both_correct": 1846, + "selected_both_wrong": 1 + }, + { + "budget": 0.18, + "threshold": -0.04530126228928566, + "operator": "ge", + "actual_fraction": 0.17177097203728361, + "metric": 0.08882240770994018, + "rescued": 8, + "harmed": 94, + "missed_rescue": 3453, + "avoided_harm": 582, + "selected_both_correct": 1961, + "selected_both_wrong": 1 + }, + { + "budget": 0.19, + "threshold": -0.04661082476377487, + "operator": "ge", + "actual_fraction": 0.1810918774966711, + "metric": 0.08900834067514127, + "rescued": 9, + "harmed": 101, + "missed_rescue": 3452, + "avoided_harm": 575, + "selected_both_correct": 2065, + "selected_both_wrong": 1 + }, + { + "budget": 0.2, + "threshold": -0.04730507358908653, + "operator": "ge", + "actual_fraction": 0.1870838881491345, + "metric": 0.09716527158339765, + "rescued": 11, + "harmed": 107, + "missed_rescue": 3450, + "avoided_harm": 569, + "selected_both_correct": 2129, + "selected_both_wrong": 1 + }, + { + "budget": 0.21, + "threshold": -0.048246145248413086, + "operator": "ge", + "actual_fraction": 0.19640479360852198, + "metric": 0.09733135745692614, + "rescued": 12, + "harmed": 112, + "missed_rescue": 3449, + "avoided_harm": 564, + "selected_both_correct": 2234, + "selected_both_wrong": 2 + }, + { + "budget": 0.22, + "threshold": -0.04900278151035309, + "operator": "ge", + "actual_fraction": 0.20838881491344874, + "metric": 0.09686185392269288, + "rescued": 14, + "harmed": 124, + "missed_rescue": 3447, + "avoided_harm": 552, + "selected_both_correct": 2363, + "selected_both_wrong": 3 + }, + { + "budget": 0.23, + "threshold": -0.050042688846588135, + "operator": "ge", + "actual_fraction": 0.21471371504660453, + "metric": 0.09673072357913137, + "rescued": 14, + "harmed": 125, + "missed_rescue": 3447, + "avoided_harm": 551, + "selected_both_correct": 2438, + "selected_both_wrong": 3 + }, + { + "budget": 0.24, + "threshold": -0.050901688635349274, + "operator": "ge", + "actual_fraction": 0.22536617842876164, + "metric": 0.10080716337703104, + "rescued": 48, + "harmed": 130, + "missed_rescue": 3413, + "avoided_harm": 546, + "selected_both_correct": 2526, + "selected_both_wrong": 4 + }, + { + "budget": 0.25, + "threshold": -0.05161922052502632, + "operator": "ge", + "actual_fraction": 0.23135818908122505, + "metric": 0.10058232286929361, + "rescued": 65, + "harmed": 134, + "missed_rescue": 3396, + "avoided_harm": 542, + "selected_both_correct": 2575, + "selected_both_wrong": 6 + }, + { + "budget": 0.26, + "threshold": -0.05245445668697357, + "operator": "ge", + "actual_fraction": 0.237017310252996, + "metric": 0.10078195333663684, + "rescued": 81, + "harmed": 136, + "missed_rescue": 3380, + "avoided_harm": 540, + "selected_both_correct": 2624, + "selected_both_wrong": 7 + }, + { + "budget": 0.27, + "threshold": -0.053079016506671906, + "operator": "ge", + "actual_fraction": 0.24167776298268975, + "metric": 0.10061353044193348, + "rescued": 95, + "harmed": 141, + "missed_rescue": 3366, + "avoided_harm": 535, + "selected_both_correct": 2660, + "selected_both_wrong": 8 + }, + { + "budget": 0.28, + "threshold": -0.053604189306497574, + "operator": "ge", + "actual_fraction": 0.2470039946737683, + "metric": 0.10058305058987213, + "rescued": 111, + "harmed": 144, + "missed_rescue": 3350, + "avoided_harm": 532, + "selected_both_correct": 2705, + "selected_both_wrong": 8 + }, + { + "budget": 0.29, + "threshold": -0.05505518987774849, + "operator": "ge", + "actual_fraction": 0.2623169107856192, + "metric": 0.10443538242821891, + "rescued": 159, + "harmed": 148, + "missed_rescue": 3302, + "avoided_harm": 528, + "selected_both_correct": 2830, + "selected_both_wrong": 15 + }, + { + "budget": 0.3, + "threshold": -0.05605518817901611, + "operator": "ge", + "actual_fraction": 0.27463382157123833, + "metric": 0.10759036763747168, + "rescued": 204, + "harmed": 158, + "missed_rescue": 3257, + "avoided_harm": 518, + "selected_both_correct": 2918, + "selected_both_wrong": 20 + }, + { + "budget": 0.31, + "threshold": -0.056986160576343536, + "operator": "ge", + "actual_fraction": 0.28462050599201066, + "metric": 0.10779091241357944, + "rescued": 236, + "harmed": 165, + "missed_rescue": 3225, + "avoided_harm": 511, + "selected_both_correct": 2997, + "selected_both_wrong": 22 + }, + { + "budget": 0.32, + "threshold": -0.0582931749522686, + "operator": "ge", + "actual_fraction": 0.29427430093209056, + "metric": 0.11066132714400925, + "rescued": 268, + "harmed": 172, + "missed_rescue": 3193, + "avoided_harm": 504, + "selected_both_correct": 3068, + "selected_both_wrong": 28 + }, + { + "budget": 0.33, + "threshold": -0.0596650205552578, + "operator": "ge", + "actual_fraction": 0.3062583222370173, + "metric": 0.11055610302472227, + "rescued": 309, + "harmed": 182, + "missed_rescue": 3152, + "avoided_harm": 494, + "selected_both_correct": 3160, + "selected_both_wrong": 29 + }, + { + "budget": 0.34, + "threshold": -0.060549296438694, + "operator": "ge", + "actual_fraction": 0.3132490013315579, + "metric": 0.1105675542193679, + "rescued": 337, + "harmed": 186, + "missed_rescue": 3124, + "avoided_harm": 490, + "selected_both_correct": 3210, + "selected_both_wrong": 31 + }, + { + "budget": 0.35, + "threshold": -0.06178095191717148, + "operator": "ge", + "actual_fraction": 0.3189081225033289, + "metric": 0.11000647533107741, + "rescued": 355, + "harmed": 194, + "missed_rescue": 3106, + "avoided_harm": 482, + "selected_both_correct": 3250, + "selected_both_wrong": 33 + }, + { + "budget": 0.36, + "threshold": -0.06328777223825455, + "operator": "ge", + "actual_fraction": 0.3318908122503329, + "metric": 0.11032470041071389, + "rescued": 397, + "harmed": 200, + "missed_rescue": 3064, + "avoided_harm": 476, + "selected_both_correct": 3350, + "selected_both_wrong": 41 + }, + { + "budget": 0.37, + "threshold": -0.0647548958659172, + "operator": "ge", + "actual_fraction": 0.3418774966711052, + "metric": 0.11048463654687253, + "rescued": 437, + "harmed": 206, + "missed_rescue": 3024, + "avoided_harm": 470, + "selected_both_correct": 3424, + "selected_both_wrong": 41 + }, + { + "budget": 0.38, + "threshold": -0.06637481600046158, + "operator": "ge", + "actual_fraction": 0.35352862849533956, + "metric": 0.11263903545823285, + "rescued": 484, + "harmed": 220, + "missed_rescue": 2977, + "avoided_harm": 456, + "selected_both_correct": 3496, + "selected_both_wrong": 48 + }, + { + "budget": 0.39, + "threshold": -0.06774259358644485, + "operator": "ge", + "actual_fraction": 0.3641810918774967, + "metric": 0.11519364722073223, + "rescued": 527, + "harmed": 227, + "missed_rescue": 2934, + "avoided_harm": 449, + "selected_both_correct": 3570, + "selected_both_wrong": 52 + }, + { + "budget": 0.4, + "threshold": -0.06843380630016327, + "operator": "ge", + "actual_fraction": 0.3698402130492676, + "metric": 0.11587206094412077, + "rescued": 550, + "harmed": 228, + "missed_rescue": 2911, + "avoided_harm": 448, + "selected_both_correct": 3609, + "selected_both_wrong": 57 + }, + { + "budget": 0.41, + "threshold": -0.0694691464304924, + "operator": "ge", + "actual_fraction": 0.3801597869507324, + "metric": 0.11650140520518415, + "rescued": 593, + "harmed": 240, + "missed_rescue": 2868, + "avoided_harm": 436, + "selected_both_correct": 3672, + "selected_both_wrong": 63 + }, + { + "budget": 0.42, + "threshold": -0.07112707942724228, + "operator": "ge", + "actual_fraction": 0.3938082556591212, + "metric": 0.11662726768380335, + "rescued": 646, + "harmed": 251, + "missed_rescue": 2815, + "avoided_harm": 425, + "selected_both_correct": 3765, + "selected_both_wrong": 70 + }, + { + "budget": 0.43, + "threshold": -0.07192998379468918, + "operator": "ge", + "actual_fraction": 0.3988015978695073, + "metric": 0.1170616446996618, + "rescued": 667, + "harmed": 252, + "missed_rescue": 2794, + "avoided_harm": 424, + "selected_both_correct": 3802, + "selected_both_wrong": 71 + }, + { + "budget": 0.44, + "threshold": -0.07266208529472351, + "operator": "ge", + "actual_fraction": 0.40279627163781623, + "metric": 0.1191343028439446, + "rescued": 682, + "harmed": 256, + "missed_rescue": 2779, + "avoided_harm": 420, + "selected_both_correct": 3830, + "selected_both_wrong": 72 + }, + { + "budget": 0.45, + "threshold": -0.0744253620505333, + "operator": "ge", + "actual_fraction": 0.41711051930758986, + "metric": 0.12016591353047154, + "rescued": 738, + "harmed": 265, + "missed_rescue": 2723, + "avoided_harm": 411, + "selected_both_correct": 3927, + "selected_both_wrong": 82 + }, + { + "budget": 0.46, + "threshold": -0.07532940804958344, + "operator": "ge", + "actual_fraction": 0.42243675099866845, + "metric": 0.12056162004139302, + "rescued": 764, + "harmed": 271, + "missed_rescue": 2697, + "avoided_harm": 405, + "selected_both_correct": 3955, + "selected_both_wrong": 86 + }, + { + "budget": 0.47, + "threshold": -0.0770750343799591, + "operator": "ge", + "actual_fraction": 0.4380825565912117, + "metric": 0.121709486901851, + "rescued": 839, + "harmed": 283, + "missed_rescue": 2622, + "avoided_harm": 393, + "selected_both_correct": 4049, + "selected_both_wrong": 93 + }, + { + "budget": 0.48, + "threshold": -0.07818037271499634, + "operator": "ge", + "actual_fraction": 0.44573901464713717, + "metric": 0.1221443001236399, + "rescued": 873, + "harmed": 288, + "missed_rescue": 2588, + "avoided_harm": 388, + "selected_both_correct": 4097, + "selected_both_wrong": 98 + }, + { + "budget": 0.49, + "threshold": -0.07998209446668625, + "operator": "ge", + "actual_fraction": 0.45905459387483355, + "metric": 0.1251607808901142, + "rescued": 928, + "harmed": 294, + "missed_rescue": 2533, + "avoided_harm": 382, + "selected_both_correct": 4185, + "selected_both_wrong": 109 + }, + { + "budget": 0.5, + "threshold": -0.08124949783086777, + "operator": "ge", + "actual_fraction": 0.46937416777629826, + "metric": 0.12551652022923962, + "rescued": 974, + "harmed": 309, + "missed_rescue": 2487, + "avoided_harm": 367, + "selected_both_correct": 4244, + "selected_both_wrong": 113 + }, + { + "budget": 0.51, + "threshold": -0.08398465067148209, + "operator": "ge", + "actual_fraction": 0.49467376830892146, + "metric": 0.12932775258830043, + "rescued": 1093, + "harmed": 334, + "missed_rescue": 2368, + "avoided_harm": 342, + "selected_both_correct": 4389, + "selected_both_wrong": 128 + }, + { + "budget": 0.52, + "threshold": -0.08537434041500092, + "operator": "ge", + "actual_fraction": 0.5056591211717709, + "metric": 0.13382418312702152, + "rescued": 1145, + "harmed": 340, + "missed_rescue": 2316, + "avoided_harm": 336, + "selected_both_correct": 4457, + "selected_both_wrong": 134 + }, + { + "budget": 0.53, + "threshold": -0.08664409816265106, + "operator": "ge", + "actual_fraction": 0.5159786950732357, + "metric": 0.1342986185420429, + "rescued": 1190, + "harmed": 350, + "missed_rescue": 2271, + "avoided_harm": 326, + "selected_both_correct": 4522, + "selected_both_wrong": 138 + }, + { + "budget": 0.54, + "threshold": -0.08777308464050293, + "operator": "ge", + "actual_fraction": 0.524300932090546, + "metric": 0.13511006926922334, + "rescued": 1228, + "harmed": 356, + "missed_rescue": 2233, + "avoided_harm": 320, + "selected_both_correct": 4574, + "selected_both_wrong": 142 + }, + { + "budget": 0.55, + "threshold": -0.08980843424797058, + "operator": "ge", + "actual_fraction": 0.5386151797603196, + "metric": 0.13883610114027645, + "rescued": 1297, + "harmed": 367, + "missed_rescue": 2164, + "avoided_harm": 309, + "selected_both_correct": 4657, + "selected_both_wrong": 151 + }, + { + "budget": 0.56, + "threshold": -0.0906188115477562, + "operator": "ge", + "actual_fraction": 0.546271637816245, + "metric": 0.13992298734058117, + "rescued": 1332, + "harmed": 375, + "missed_rescue": 2129, + "avoided_harm": 301, + "selected_both_correct": 4702, + "selected_both_wrong": 155 + }, + { + "budget": 0.57, + "threshold": -0.09160211682319641, + "operator": "ge", + "actual_fraction": 0.5555925432756325, + "metric": 0.14072696062294485, + "rescued": 1379, + "harmed": 383, + "missed_rescue": 2082, + "avoided_harm": 293, + "selected_both_correct": 4751, + "selected_both_wrong": 163 + }, + { + "budget": 0.58, + "threshold": -0.09283093363046646, + "operator": "ge", + "actual_fraction": 0.568242343541944, + "metric": 0.1472561682256289, + "rescued": 1434, + "harmed": 393, + "missed_rescue": 2027, + "avoided_harm": 283, + "selected_both_correct": 4827, + "selected_both_wrong": 174 + }, + { + "budget": 0.59, + "threshold": -0.0941123366355896, + "operator": "ge", + "actual_fraction": 0.5778961384820239, + "metric": 0.15078547139034337, + "rescued": 1482, + "harmed": 399, + "missed_rescue": 1979, + "avoided_harm": 277, + "selected_both_correct": 4882, + "selected_both_wrong": 181 + }, + { + "budget": 0.6, + "threshold": -0.09614869952201843, + "operator": "ge", + "actual_fraction": 0.5878828229027963, + "metric": 0.15176768468059784, + "rescued": 1528, + "harmed": 406, + "missed_rescue": 1933, + "avoided_harm": 270, + "selected_both_correct": 4939, + "selected_both_wrong": 191 + }, + { + "budget": 0.61, + "threshold": -0.09772664308547974, + "operator": "ge", + "actual_fraction": 0.5975366178428761, + "metric": 0.1532082936739425, + "rescued": 1567, + "harmed": 414, + "missed_rescue": 1894, + "avoided_harm": 262, + "selected_both_correct": 4998, + "selected_both_wrong": 201 + }, + { + "budget": 0.62, + "threshold": -0.09911532700061798, + "operator": "ge", + "actual_fraction": 0.6081890812250332, + "metric": 0.15424732264646662, + "rescued": 1615, + "harmed": 422, + "missed_rescue": 1846, + "avoided_harm": 254, + "selected_both_correct": 5061, + "selected_both_wrong": 210 + }, + { + "budget": 0.63, + "threshold": -0.10007988661527634, + "operator": "ge", + "actual_fraction": 0.6161784287616512, + "metric": 0.1548871102093026, + "rescued": 1652, + "harmed": 430, + "missed_rescue": 1809, + "avoided_harm": 246, + "selected_both_correct": 5104, + "selected_both_wrong": 218 + }, + { + "budget": 0.64, + "threshold": -0.10177135467529297, + "operator": "ge", + "actual_fraction": 0.6278295605858855, + "metric": 0.15779981350208042, + "rescued": 1709, + "harmed": 438, + "missed_rescue": 1752, + "avoided_harm": 238, + "selected_both_correct": 5169, + "selected_both_wrong": 228 + }, + { + "budget": 0.65, + "threshold": -0.10395076125860214, + "operator": "ge", + "actual_fraction": 0.6451398135818908, + "metric": 0.16144984235200888, + "rescued": 1795, + "harmed": 448, + "missed_rescue": 1666, + "avoided_harm": 228, + "selected_both_correct": 5271, + "selected_both_wrong": 238 + }, + { + "budget": 0.66, + "threshold": -0.10524322092533112, + "operator": "ge", + "actual_fraction": 0.6491344873501997, + "metric": 0.16469010323619201, + "rescued": 1816, + "harmed": 448, + "missed_rescue": 1645, + "avoided_harm": 228, + "selected_both_correct": 5295, + "selected_both_wrong": 241 + }, + { + "budget": 0.67, + "threshold": -0.10689426958560944, + "operator": "ge", + "actual_fraction": 0.6601198402130493, + "metric": 0.16600232483815708, + "rescued": 1872, + "harmed": 456, + "missed_rescue": 1589, + "avoided_harm": 220, + "selected_both_correct": 5357, + "selected_both_wrong": 247 + }, + { + "budget": 0.68, + "threshold": -0.11014843732118607, + "operator": "ge", + "actual_fraction": 0.6754327563249002, + "metric": 0.16941529572817854, + "rescued": 1941, + "harmed": 465, + "missed_rescue": 1520, + "avoided_harm": 211, + "selected_both_correct": 5452, + "selected_both_wrong": 258 + }, + { + "budget": 0.69, + "threshold": -0.11117545515298843, + "operator": "ge", + "actual_fraction": 0.6790945406125166, + "metric": 0.17007927366959905, + "rescued": 1960, + "harmed": 467, + "missed_rescue": 1501, + "avoided_harm": 209, + "selected_both_correct": 5471, + "selected_both_wrong": 262 + }, + { + "budget": 0.7, + "threshold": -0.11230025440454483, + "operator": "ge", + "actual_fraction": 0.6870838881491345, + "metric": 0.17172535263617786, + "rescued": 2005, + "harmed": 471, + "missed_rescue": 1456, + "avoided_harm": 205, + "selected_both_correct": 5514, + "selected_both_wrong": 266 + }, + { + "budget": 0.71, + "threshold": -0.11423606425523758, + "operator": "ge", + "actual_fraction": 0.6960719041278296, + "metric": 0.17427123861822869, + "rescued": 2047, + "harmed": 477, + "missed_rescue": 1414, + "avoided_harm": 199, + "selected_both_correct": 5562, + "selected_both_wrong": 278 + }, + { + "budget": 0.72, + "threshold": -0.11647249013185501, + "operator": "ge", + "actual_fraction": 0.7053928095872171, + "metric": 0.17661848328954455, + "rescued": 2090, + "harmed": 486, + "missed_rescue": 1371, + "avoided_harm": 190, + "selected_both_correct": 5613, + "selected_both_wrong": 287 + }, + { + "budget": 0.73, + "threshold": -0.11962417513132095, + "operator": "ge", + "actual_fraction": 0.7193741677762983, + "metric": 0.18093016254845928, + "rescued": 2156, + "harmed": 500, + "missed_rescue": 1305, + "avoided_harm": 176, + "selected_both_correct": 5687, + "selected_both_wrong": 301 + }, + { + "budget": 0.74, + "threshold": -0.12126146256923676, + "operator": "ge", + "actual_fraction": 0.7250332889480693, + "metric": 0.18214911344128, + "rescued": 2186, + "harmed": 504, + "missed_rescue": 1275, + "avoided_harm": 172, + "selected_both_correct": 5719, + "selected_both_wrong": 303 + }, + { + "budget": 0.75, + "threshold": -0.1230982318520546, + "operator": "ge", + "actual_fraction": 0.7366844207723036, + "metric": 0.18474146474609127, + "rescued": 2239, + "harmed": 510, + "missed_rescue": 1222, + "avoided_harm": 166, + "selected_both_correct": 5790, + "selected_both_wrong": 313 + }, + { + "budget": 0.76, + "threshold": -0.12517565488815308, + "operator": "ge", + "actual_fraction": 0.7446737683089214, + "metric": 0.1868238104649864, + "rescued": 2278, + "harmed": 513, + "missed_rescue": 1183, + "avoided_harm": 163, + "selected_both_correct": 5841, + "selected_both_wrong": 316 + }, + { + "budget": 0.77, + "threshold": -0.12628257274627686, + "operator": "ge", + "actual_fraction": 0.7480026631158455, + "metric": 0.18780967278670027, + "rescued": 2296, + "harmed": 513, + "missed_rescue": 1165, + "avoided_harm": 163, + "selected_both_correct": 5861, + "selected_both_wrong": 318 + }, + { + "budget": 0.78, + "threshold": -0.12975578010082245, + "operator": "ge", + "actual_fraction": 0.7603195739014648, + "metric": 0.19033940523013337, + "rescued": 2354, + "harmed": 518, + "missed_rescue": 1107, + "avoided_harm": 158, + "selected_both_correct": 5931, + "selected_both_wrong": 333 + }, + { + "budget": 0.79, + "threshold": -0.13229140639305115, + "operator": "ge", + "actual_fraction": 0.7713049267643143, + "metric": 0.19289795000814608, + "rescued": 2407, + "harmed": 524, + "missed_rescue": 1054, + "avoided_harm": 152, + "selected_both_correct": 5996, + "selected_both_wrong": 341 + }, + { + "budget": 0.8, + "threshold": -0.1373685747385025, + "operator": "ge", + "actual_fraction": 0.7909454061251664, + "metric": 0.1970331406058424, + "rescued": 2510, + "harmed": 548, + "missed_rescue": 951, + "avoided_harm": 128, + "selected_both_correct": 6090, + "selected_both_wrong": 356 + }, + { + "budget": 0.81, + "threshold": -0.1407174915075302, + "operator": "ge", + "actual_fraction": 0.8012649800266312, + "metric": 0.1991542632866029, + "rescued": 2560, + "harmed": 556, + "missed_rescue": 901, + "avoided_harm": 120, + "selected_both_correct": 6144, + "selected_both_wrong": 368 + }, + { + "budget": 0.82, + "threshold": -0.1433146893978119, + "operator": "ge", + "actual_fraction": 0.8125832223701731, + "metric": 0.2006018605686543, + "rescued": 2613, + "harmed": 569, + "missed_rescue": 848, + "avoided_harm": 107, + "selected_both_correct": 6204, + "selected_both_wrong": 378 + }, + { + "budget": 0.83, + "threshold": -0.1457100659608841, + "operator": "ge", + "actual_fraction": 0.8199067909454061, + "metric": 0.20192079535680077, + "rescued": 2654, + "harmed": 577, + "missed_rescue": 807, + "avoided_harm": 99, + "selected_both_correct": 6238, + "selected_both_wrong": 383 + }, + { + "budget": 0.84, + "threshold": -0.148710235953331, + "operator": "ge", + "actual_fraction": 0.8242343541944075, + "metric": 0.20317936971986447, + "rescued": 2674, + "harmed": 578, + "missed_rescue": 787, + "avoided_harm": 98, + "selected_both_correct": 6264, + "selected_both_wrong": 388 + }, + { + "budget": 0.85, + "threshold": -0.15291842818260193, + "operator": "ge", + "actual_fraction": 0.8338881491344874, + "metric": 0.2060155880710891, + "rescued": 2727, + "harmed": 591, + "missed_rescue": 734, + "avoided_harm": 85, + "selected_both_correct": 6311, + "selected_both_wrong": 391 + }, + { + "budget": 0.86, + "threshold": -0.15889839828014374, + "operator": "ge", + "actual_fraction": 0.8458721704394141, + "metric": 0.20894280280516728, + "rescued": 2777, + "harmed": 600, + "missed_rescue": 684, + "avoided_harm": 76, + "selected_both_correct": 6383, + "selected_both_wrong": 404 + }, + { + "budget": 0.87, + "threshold": -0.16629797220230103, + "operator": "ge", + "actual_fraction": 0.8585219707057257, + "metric": 0.2146238532926011, + "rescued": 2839, + "harmed": 603, + "missed_rescue": 622, + "avoided_harm": 73, + "selected_both_correct": 6459, + "selected_both_wrong": 415 + }, + { + "budget": 0.88, + "threshold": -0.17295491695404053, + "operator": "ge", + "actual_fraction": 0.8655126498002663, + "metric": 0.21521435730631172, + "rescued": 2868, + "harmed": 611, + "missed_rescue": 593, + "avoided_harm": 65, + "selected_both_correct": 6497, + "selected_both_wrong": 424 + }, + { + "budget": 0.89, + "threshold": -0.18053314089775085, + "operator": "ge", + "actual_fraction": 0.8738348868175766, + "metric": 0.2202221543306381, + "rescued": 2899, + "harmed": 617, + "missed_rescue": 562, + "avoided_harm": 59, + "selected_both_correct": 6544, + "selected_both_wrong": 440 + }, + { + "budget": 0.9, + "threshold": -0.19442276656627655, + "operator": "ge", + "actual_fraction": 0.8878162450066578, + "metric": 0.22393167777841913, + "rescued": 2961, + "harmed": 626, + "missed_rescue": 500, + "avoided_harm": 50, + "selected_both_correct": 6624, + "selected_both_wrong": 457 + }, + { + "budget": 0.91, + "threshold": -0.20900504291057587, + "operator": "ge", + "actual_fraction": 0.8961384820239681, + "metric": 0.22795997864890608, + "rescued": 3002, + "harmed": 629, + "missed_rescue": 459, + "avoided_harm": 47, + "selected_both_correct": 6676, + "selected_both_wrong": 461 + }, + { + "budget": 0.92, + "threshold": -0.22383525967597961, + "operator": "ge", + "actual_fraction": 0.9027962716378163, + "metric": 0.23118004657749075, + "rescued": 3034, + "harmed": 634, + "missed_rescue": 427, + "avoided_harm": 42, + "selected_both_correct": 6715, + "selected_both_wrong": 465 + }, + { + "budget": 0.93, + "threshold": -0.25952714681625366, + "operator": "ge", + "actual_fraction": 0.9184420772303595, + "metric": 0.23697523764758885, + "rescued": 3106, + "harmed": 646, + "missed_rescue": 355, + "avoided_harm": 30, + "selected_both_correct": 6801, + "selected_both_wrong": 483 + }, + { + "budget": 0.94, + "threshold": -0.2804492115974426, + "operator": "ge", + "actual_fraction": 0.9257656458055925, + "metric": 0.24091409959663834, + "rescued": 3141, + "harmed": 647, + "missed_rescue": 320, + "avoided_harm": 29, + "selected_both_correct": 6844, + "selected_both_wrong": 492 + }, + { + "budget": 0.95, + "threshold": -0.2984963357448578, + "operator": "ge", + "actual_fraction": 0.9320905459387483, + "metric": 0.2426031328046549, + "rescued": 3163, + "harmed": 648, + "missed_rescue": 298, + "avoided_harm": 28, + "selected_both_correct": 6890, + "selected_both_wrong": 499 + }, + { + "budget": 0.96, + "threshold": -0.32223379611968994, + "operator": "ge", + "actual_fraction": 0.9410785619174434, + "metric": 0.24537310927695588, + "rescued": 3203, + "harmed": 656, + "missed_rescue": 258, + "avoided_harm": 20, + "selected_both_correct": 6944, + "selected_both_wrong": 505 + }, + { + "budget": 0.97, + "threshold": -0.36503666639328003, + "operator": "ge", + "actual_fraction": 0.9573901464713716, + "metric": 0.25207284548974734, + "rescued": 3277, + "harmed": 663, + "missed_rescue": 184, + "avoided_harm": 13, + "selected_both_correct": 7042, + "selected_both_wrong": 522 + }, + { + "budget": 0.98, + "threshold": -0.4221048653125763, + "operator": "ge", + "actual_fraction": 0.9790279627163782, + "metric": 0.2631165680871859, + "rescued": 3373, + "harmed": 668, + "missed_rescue": 88, + "avoided_harm": 8, + "selected_both_correct": 7186, + "selected_both_wrong": 537 + }, + { + "budget": 0.99, + "threshold": -0.44162338972091675, + "operator": "ge", + "actual_fraction": 0.9860186418109188, + "metric": 0.26818431283513144, + "rescued": 3404, + "harmed": 671, + "missed_rescue": 57, + "avoided_harm": 5, + "selected_both_correct": 7228, + "selected_both_wrong": 545 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.27001009067886245, + "rescued": 3461, + "harmed": 676, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 7329, + "selected_both_wrong": 550 + } + ] + }, + "learned": { + "selected_budget_index": 83, + "validation_area_0_25": 0.13205087205171664, + "validation_point": { + "budget": 0.83, + "threshold": -0.4216182231903076, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.2719739402647948, + "rescued": 814, + "harmed": 149, + "missed_rescue": 1, + "avoided_harm": 36, + "selected_both_correct": 1238, + "selected_both_wrong": 123 + }, + "test_point": { + "budget": 0.83, + "threshold": -0.4216182231903076, + "operator": "ge", + "actual_fraction": 0.831225033288948, + "metric": 0.26693203615113315, + "rescued": 3448, + "harmed": 576, + "missed_rescue": 13, + "avoided_harm": 100, + "selected_both_correct": 5415, + "selected_both_wrong": 549 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.07641657905799261, + "rescued": 0, + "harmed": 0, + "missed_rescue": 815, + "avoided_harm": 185, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.754962205886841, + "operator": "ge", + "actual_fraction": 0.009985734664764621, + "metric": 0.07723912349750027, + "rescued": 14, + "harmed": 1, + "missed_rescue": 801, + "avoided_harm": 184, + "selected_both_correct": 12, + "selected_both_wrong": 1 + }, + { + "budget": 0.02, + "threshold": 2.6711103916168213, + "operator": "ge", + "actual_fraction": 0.019971469329529243, + "metric": 0.11132271148956185, + "rescued": 26, + "harmed": 1, + "missed_rescue": 789, + "avoided_harm": 184, + "selected_both_correct": 27, + "selected_both_wrong": 2 + }, + { + "budget": 0.03, + "threshold": 2.6189634799957275, + "operator": "ge", + "actual_fraction": 0.029957203994293864, + "metric": 0.10991785946649317, + "rescued": 37, + "harmed": 2, + "missed_rescue": 778, + "avoided_harm": 183, + "selected_both_correct": 41, + "selected_both_wrong": 4 + }, + { + "budget": 0.04, + "threshold": 2.5298099517822266, + "operator": "ge", + "actual_fraction": 0.039942938659058486, + "metric": 0.13539632997572107, + "rescued": 48, + "harmed": 4, + "missed_rescue": 767, + "avoided_harm": 181, + "selected_both_correct": 50, + "selected_both_wrong": 10 + }, + { + "budget": 0.05, + "threshold": 2.486783981323242, + "operator": "ge", + "actual_fraction": 0.04992867332382311, + "metric": 0.13635451200120616, + "rescued": 61, + "harmed": 4, + "missed_rescue": 754, + "avoided_harm": 181, + "selected_both_correct": 64, + "selected_both_wrong": 11 + }, + { + "budget": 0.06, + "threshold": 2.4111104011535645, + "operator": "ge", + "actual_fraction": 0.05991440798858773, + "metric": 0.13735988780741779, + "rescued": 74, + "harmed": 4, + "missed_rescue": 741, + "avoided_harm": 181, + "selected_both_correct": 79, + "selected_both_wrong": 11 + }, + { + "budget": 0.07, + "threshold": 2.3550784587860107, + "operator": "ge", + "actual_fraction": 0.06990014265335236, + "metric": 0.1351633110012043, + "rescued": 87, + "harmed": 5, + "missed_rescue": 728, + "avoided_harm": 180, + "selected_both_correct": 93, + "selected_both_wrong": 11 + }, + { + "budget": 0.08, + "threshold": 2.321181297302246, + "operator": "ge", + "actual_fraction": 0.07988587731811697, + "metric": 0.13577321934823655, + "rescued": 94, + "harmed": 5, + "missed_rescue": 721, + "avoided_harm": 180, + "selected_both_correct": 112, + "selected_both_wrong": 13 + }, + { + "budget": 0.09, + "threshold": 2.26045823097229, + "operator": "ge", + "actual_fraction": 0.0898716119828816, + "metric": 0.1338396123999839, + "rescued": 107, + "harmed": 7, + "missed_rescue": 708, + "avoided_harm": 178, + "selected_both_correct": 124, + "selected_both_wrong": 14 + }, + { + "budget": 0.1, + "threshold": 2.2216036319732666, + "operator": "ge", + "actual_fraction": 0.09985734664764621, + "metric": 0.13490375448424985, + "rescued": 120, + "harmed": 7, + "missed_rescue": 695, + "avoided_harm": 178, + "selected_both_correct": 137, + "selected_both_wrong": 16 + }, + { + "budget": 0.11, + "threshold": 2.1819043159484863, + "operator": "ge", + "actual_fraction": 0.10984308131241084, + "metric": 0.13304780973182867, + "rescued": 132, + "harmed": 9, + "missed_rescue": 683, + "avoided_harm": 176, + "selected_both_correct": 147, + "selected_both_wrong": 20 + }, + { + "budget": 0.12, + "threshold": 2.16599702835083, + "operator": "ge", + "actual_fraction": 0.11982881597717546, + "metric": 0.13068814302580536, + "rescued": 145, + "harmed": 11, + "missed_rescue": 670, + "avoided_harm": 174, + "selected_both_correct": 159, + "selected_both_wrong": 21 + }, + { + "budget": 0.13, + "threshold": 2.1376547813415527, + "operator": "ge", + "actual_fraction": 0.12981455064194009, + "metric": 0.15122753646913312, + "rescued": 158, + "harmed": 12, + "missed_rescue": 657, + "avoided_harm": 173, + "selected_both_correct": 171, + "selected_both_wrong": 23 + }, + { + "budget": 0.14, + "threshold": 2.1090269088745117, + "operator": "ge", + "actual_fraction": 0.1398002853067047, + "metric": 0.1464405791306376, + "rescued": 170, + "harmed": 15, + "missed_rescue": 645, + "avoided_harm": 170, + "selected_both_correct": 182, + "selected_both_wrong": 25 + }, + { + "budget": 0.15, + "threshold": 2.0827982425689697, + "operator": "ge", + "actual_fraction": 0.14978601997146934, + "metric": 0.14540766103266103, + "rescued": 184, + "harmed": 16, + "missed_rescue": 631, + "avoided_harm": 169, + "selected_both_correct": 193, + "selected_both_wrong": 27 + }, + { + "budget": 0.16, + "threshold": 2.0538668632507324, + "operator": "ge", + "actual_fraction": 0.15977175463623394, + "metric": 0.14048938381429565, + "rescued": 196, + "harmed": 19, + "missed_rescue": 619, + "avoided_harm": 166, + "selected_both_correct": 203, + "selected_both_wrong": 30 + }, + { + "budget": 0.17, + "threshold": 2.027881145477295, + "operator": "ge", + "actual_fraction": 0.16975748930099857, + "metric": 0.13829881832871008, + "rescued": 208, + "harmed": 21, + "missed_rescue": 607, + "avoided_harm": 164, + "selected_both_correct": 212, + "selected_both_wrong": 35 + }, + { + "budget": 0.18, + "threshold": 2.0083131790161133, + "operator": "ge", + "actual_fraction": 0.1797432239657632, + "metric": 0.1380118600527366, + "rescued": 220, + "harmed": 22, + "missed_rescue": 595, + "avoided_harm": 163, + "selected_both_correct": 226, + "selected_both_wrong": 36 + }, + { + "budget": 0.19, + "threshold": 1.978614091873169, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.13903385537198398, + "rescued": 231, + "harmed": 23, + "missed_rescue": 584, + "avoided_harm": 162, + "selected_both_correct": 240, + "selected_both_wrong": 38 + }, + { + "budget": 0.2, + "threshold": 1.968393087387085, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.13712335572945195, + "rescued": 242, + "harmed": 26, + "missed_rescue": 573, + "avoided_harm": 159, + "selected_both_correct": 253, + "selected_both_wrong": 39 + }, + { + "budget": 0.21, + "threshold": 1.9526315927505493, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.13708380294918757, + "rescued": 256, + "harmed": 28, + "missed_rescue": 559, + "avoided_harm": 157, + "selected_both_correct": 263, + "selected_both_wrong": 41 + }, + { + "budget": 0.22, + "threshold": 1.936199426651001, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.13707729953670883, + "rescued": 269, + "harmed": 29, + "missed_rescue": 546, + "avoided_harm": 156, + "selected_both_correct": 275, + "selected_both_wrong": 43 + }, + { + "budget": 0.23, + "threshold": 1.9114916324615479, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.1365203296199742, + "rescued": 277, + "harmed": 31, + "missed_rescue": 538, + "avoided_harm": 154, + "selected_both_correct": 287, + "selected_both_wrong": 49 + }, + { + "budget": 0.24, + "threshold": 1.9026963710784912, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.13677830533885202, + "rescued": 287, + "harmed": 32, + "missed_rescue": 528, + "avoided_harm": 153, + "selected_both_correct": 303, + "selected_both_wrong": 50 + }, + { + "budget": 0.25, + "threshold": 1.8903357982635498, + "operator": "ge", + "actual_fraction": 0.24964336661911554, + "metric": 0.1365135749640358, + "rescued": 300, + "harmed": 37, + "missed_rescue": 515, + "avoided_harm": 148, + "selected_both_correct": 311, + "selected_both_wrong": 52 + }, + { + "budget": 0.26, + "threshold": 1.8735990524291992, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.13809157884465262, + "rescued": 314, + "harmed": 37, + "missed_rescue": 501, + "avoided_harm": 148, + "selected_both_correct": 324, + "selected_both_wrong": 53 + }, + { + "budget": 0.27, + "threshold": 1.8593947887420654, + "operator": "ge", + "actual_fraction": 0.26961483594864477, + "metric": 0.13972534273912546, + "rescued": 328, + "harmed": 37, + "missed_rescue": 487, + "avoided_harm": 148, + "selected_both_correct": 336, + "selected_both_wrong": 55 + }, + { + "budget": 0.28, + "threshold": 1.841538429260254, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.140080848156341, + "rescued": 339, + "harmed": 38, + "missed_rescue": 476, + "avoided_harm": 147, + "selected_both_correct": 351, + "selected_both_wrong": 56 + }, + { + "budget": 0.29, + "threshold": 1.8348605632781982, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.1383045282600874, + "rescued": 347, + "harmed": 41, + "missed_rescue": 468, + "avoided_harm": 144, + "selected_both_correct": 364, + "selected_both_wrong": 60 + }, + { + "budget": 0.3, + "threshold": 1.8283830881118774, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.13908996416407296, + "rescued": 354, + "harmed": 42, + "missed_rescue": 461, + "avoided_harm": 143, + "selected_both_correct": 382, + "selected_both_wrong": 62 + }, + { + "budget": 0.31, + "threshold": 1.8149042129516602, + "operator": "ge", + "actual_fraction": 0.3095577746077033, + "metric": 0.13992285852383174, + "rescued": 366, + "harmed": 43, + "missed_rescue": 449, + "avoided_harm": 142, + "selected_both_correct": 395, + "selected_both_wrong": 64 + }, + { + "budget": 0.32, + "threshold": 1.8025038242340088, + "operator": "ge", + "actual_fraction": 0.3195435092724679, + "metric": 0.13947831572129712, + "rescued": 380, + "harmed": 46, + "missed_rescue": 435, + "avoided_harm": 139, + "selected_both_correct": 404, + "selected_both_wrong": 66 + }, + { + "budget": 0.33, + "threshold": 1.7916626930236816, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.14129537608304688, + "rescued": 394, + "harmed": 46, + "missed_rescue": 421, + "avoided_harm": 139, + "selected_both_correct": 417, + "selected_both_wrong": 67 + }, + { + "budget": 0.34, + "threshold": 1.7744719982147217, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.14052029330717852, + "rescued": 408, + "harmed": 50, + "missed_rescue": 407, + "avoided_harm": 135, + "selected_both_correct": 426, + "selected_both_wrong": 68 + }, + { + "budget": 0.35, + "threshold": 1.7519283294677734, + "operator": "ge", + "actual_fraction": 0.34950071326676174, + "metric": 0.14912382390035073, + "rescued": 421, + "harmed": 54, + "missed_rescue": 394, + "avoided_harm": 131, + "selected_both_correct": 435, + "selected_both_wrong": 70 + }, + { + "budget": 0.36, + "threshold": 1.743757963180542, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.15069715306057904, + "rescued": 433, + "harmed": 55, + "missed_rescue": 382, + "avoided_harm": 130, + "selected_both_correct": 449, + "selected_both_wrong": 71 + }, + { + "budget": 0.37, + "threshold": 1.7330536842346191, + "operator": "ge", + "actual_fraction": 0.369472182596291, + "metric": 0.15204625696308266, + "rescued": 443, + "harmed": 55, + "missed_rescue": 372, + "avoided_harm": 130, + "selected_both_correct": 467, + "selected_both_wrong": 71 + }, + { + "budget": 0.38, + "threshold": 1.7175750732421875, + "operator": "ge", + "actual_fraction": 0.37945791726105566, + "metric": 0.16059867503338637, + "rescued": 455, + "harmed": 57, + "missed_rescue": 360, + "avoided_harm": 128, + "selected_both_correct": 477, + "selected_both_wrong": 75 + }, + { + "budget": 0.39, + "threshold": 1.7006492614746094, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.1612074726298864, + "rescued": 467, + "harmed": 60, + "missed_rescue": 348, + "avoided_harm": 125, + "selected_both_correct": 488, + "selected_both_wrong": 77 + }, + { + "budget": 0.4, + "threshold": 1.6880998611450195, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.1622979339751485, + "rescued": 478, + "harmed": 62, + "missed_rescue": 337, + "avoided_harm": 123, + "selected_both_correct": 502, + "selected_both_wrong": 78 + }, + { + "budget": 0.41, + "threshold": 1.6769726276397705, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.16374903924765133, + "rescued": 490, + "harmed": 63, + "missed_rescue": 325, + "avoided_harm": 122, + "selected_both_correct": 517, + "selected_both_wrong": 78 + }, + { + "budget": 0.42, + "threshold": 1.647587537765503, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.16457308284876496, + "rescued": 497, + "harmed": 64, + "missed_rescue": 318, + "avoided_harm": 121, + "selected_both_correct": 535, + "selected_both_wrong": 80 + }, + { + "budget": 0.43, + "threshold": 1.6429295539855957, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.17293453801349049, + "rescued": 507, + "harmed": 66, + "missed_rescue": 308, + "avoided_harm": 119, + "selected_both_correct": 549, + "selected_both_wrong": 82 + }, + { + "budget": 0.44, + "threshold": 1.6297134160995483, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.1734501310279295, + "rescued": 517, + "harmed": 67, + "missed_rescue": 298, + "avoided_harm": 118, + "selected_both_correct": 565, + "selected_both_wrong": 83 + }, + { + "budget": 0.45, + "threshold": 1.6237554550170898, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.17559975317560655, + "rescued": 532, + "harmed": 68, + "missed_rescue": 283, + "avoided_harm": 117, + "selected_both_correct": 576, + "selected_both_wrong": 84 + }, + { + "budget": 0.46, + "threshold": 1.6143747568130493, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.17796417028408554, + "rescued": 541, + "harmed": 71, + "missed_rescue": 274, + "avoided_harm": 114, + "selected_both_correct": 590, + "selected_both_wrong": 86 + }, + { + "budget": 0.47, + "threshold": 1.5888383388519287, + "operator": "ge", + "actual_fraction": 0.4693295292439372, + "metric": 0.17374457286416198, + "rescued": 547, + "harmed": 74, + "missed_rescue": 268, + "avoided_harm": 111, + "selected_both_correct": 609, + "selected_both_wrong": 86 + }, + { + "budget": 0.48, + "threshold": 1.5711208581924438, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.18029615731155585, + "rescued": 559, + "harmed": 77, + "missed_rescue": 256, + "avoided_harm": 108, + "selected_both_correct": 621, + "selected_both_wrong": 87 + }, + { + "budget": 0.49, + "threshold": 1.5494966506958008, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.18172527158376214, + "rescued": 572, + "harmed": 80, + "missed_rescue": 243, + "avoided_harm": 105, + "selected_both_correct": 633, + "selected_both_wrong": 87 + }, + { + "budget": 0.5, + "threshold": 1.5336257219314575, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.18220510426392778, + "rescued": 580, + "harmed": 82, + "missed_rescue": 235, + "avoided_harm": 103, + "selected_both_correct": 649, + "selected_both_wrong": 89 + }, + { + "budget": 0.51, + "threshold": 1.5163730382919312, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.18305036355778223, + "rescued": 588, + "harmed": 86, + "missed_rescue": 227, + "avoided_harm": 99, + "selected_both_correct": 663, + "selected_both_wrong": 91 + }, + { + "budget": 0.52, + "threshold": 1.5053234100341797, + "operator": "ge", + "actual_fraction": 0.5192582025677603, + "metric": 0.18380537400145244, + "rescued": 597, + "harmed": 88, + "missed_rescue": 218, + "avoided_harm": 97, + "selected_both_correct": 678, + "selected_both_wrong": 93 + }, + { + "budget": 0.53, + "threshold": 1.4833742380142212, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.1854132152963029, + "rescued": 606, + "harmed": 89, + "missed_rescue": 209, + "avoided_harm": 96, + "selected_both_correct": 691, + "selected_both_wrong": 98 + }, + { + "budget": 0.54, + "threshold": 1.4717422723770142, + "operator": "ge", + "actual_fraction": 0.5392296718972895, + "metric": 0.18635960019011744, + "rescued": 614, + "harmed": 91, + "missed_rescue": 201, + "avoided_harm": 94, + "selected_both_correct": 707, + "selected_both_wrong": 100 + }, + { + "budget": 0.55, + "threshold": 1.4611992835998535, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.18918646125850463, + "rescued": 624, + "harmed": 91, + "missed_rescue": 191, + "avoided_harm": 94, + "selected_both_correct": 724, + "selected_both_wrong": 101 + }, + { + "budget": 0.56, + "threshold": 1.444828748703003, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.1912258831351297, + "rescued": 632, + "harmed": 91, + "missed_rescue": 183, + "avoided_harm": 94, + "selected_both_correct": 743, + "selected_both_wrong": 102 + }, + { + "budget": 0.57, + "threshold": 1.4202982187271118, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.19325962167945576, + "rescued": 642, + "harmed": 93, + "missed_rescue": 173, + "avoided_harm": 92, + "selected_both_correct": 757, + "selected_both_wrong": 104 + }, + { + "budget": 0.58, + "threshold": 1.402458667755127, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.19583007756717632, + "rescued": 655, + "harmed": 95, + "missed_rescue": 160, + "avoided_harm": 90, + "selected_both_correct": 770, + "selected_both_wrong": 104 + }, + { + "budget": 0.59, + "threshold": 1.3859935998916626, + "operator": "ge", + "actual_fraction": 0.5891583452211127, + "metric": 0.19835925590076375, + "rescued": 665, + "harmed": 96, + "missed_rescue": 150, + "avoided_harm": 89, + "selected_both_correct": 786, + "selected_both_wrong": 105 + }, + { + "budget": 0.6, + "threshold": 1.3550105094909668, + "operator": "ge", + "actual_fraction": 0.5991440798858774, + "metric": 0.20176133587955766, + "rescued": 675, + "harmed": 99, + "missed_rescue": 140, + "avoided_harm": 86, + "selected_both_correct": 799, + "selected_both_wrong": 107 + }, + { + "budget": 0.61, + "threshold": 1.336022973060608, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.20984714332841836, + "rescued": 686, + "harmed": 101, + "missed_rescue": 129, + "avoided_harm": 84, + "selected_both_correct": 812, + "selected_both_wrong": 109 + }, + { + "budget": 0.62, + "threshold": 1.2876667976379395, + "operator": "ge", + "actual_fraction": 0.6191155492154066, + "metric": 0.21195483107247814, + "rescued": 695, + "harmed": 102, + "missed_rescue": 120, + "avoided_harm": 83, + "selected_both_correct": 828, + "selected_both_wrong": 111 + }, + { + "budget": 0.63, + "threshold": 1.2704218626022339, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.21986405697321376, + "rescued": 706, + "harmed": 104, + "missed_rescue": 109, + "avoided_harm": 81, + "selected_both_correct": 842, + "selected_both_wrong": 112 + }, + { + "budget": 0.64, + "threshold": 1.2135732173919678, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.22155152807391612, + "rescued": 712, + "harmed": 105, + "missed_rescue": 103, + "avoided_harm": 80, + "selected_both_correct": 859, + "selected_both_wrong": 116 + }, + { + "budget": 0.65, + "threshold": 1.1857013702392578, + "operator": "ge", + "actual_fraction": 0.6490727532097005, + "metric": 0.23011338926520924, + "rescued": 723, + "harmed": 106, + "missed_rescue": 92, + "avoided_harm": 79, + "selected_both_correct": 874, + "selected_both_wrong": 117 + }, + { + "budget": 0.66, + "threshold": 1.1139411926269531, + "operator": "ge", + "actual_fraction": 0.6590584878744651, + "metric": 0.23426863099136824, + "rescued": 734, + "harmed": 107, + "missed_rescue": 81, + "avoided_harm": 78, + "selected_both_correct": 890, + "selected_both_wrong": 117 + }, + { + "budget": 0.67, + "threshold": 1.0512058734893799, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.2349627516789209, + "rescued": 743, + "harmed": 113, + "missed_rescue": 72, + "avoided_harm": 72, + "selected_both_correct": 902, + "selected_both_wrong": 118 + }, + { + "budget": 0.68, + "threshold": 1.0203181505203247, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.236929710543952, + "rescued": 751, + "harmed": 117, + "missed_rescue": 64, + "avoided_harm": 68, + "selected_both_correct": 917, + "selected_both_wrong": 119 + }, + { + "budget": 0.69, + "threshold": 0.980606198310852, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.24032594086021508, + "rescued": 757, + "harmed": 117, + "missed_rescue": 58, + "avoided_harm": 68, + "selected_both_correct": 938, + "selected_both_wrong": 120 + }, + { + "budget": 0.7, + "threshold": 0.9318755269050598, + "operator": "ge", + "actual_fraction": 0.6990014265335235, + "metric": 0.2433066793582963, + "rescued": 763, + "harmed": 118, + "missed_rescue": 52, + "avoided_harm": 67, + "selected_both_correct": 959, + "selected_both_wrong": 120 + }, + { + "budget": 0.71, + "threshold": 0.854070782661438, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.24643409860836926, + "rescued": 771, + "harmed": 121, + "missed_rescue": 44, + "avoided_harm": 64, + "selected_both_correct": 976, + "selected_both_wrong": 120 + }, + { + "budget": 0.72, + "threshold": 0.8128365278244019, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.24680302827672806, + "rescued": 777, + "harmed": 125, + "missed_rescue": 38, + "avoided_harm": 60, + "selected_both_correct": 994, + "selected_both_wrong": 120 + }, + { + "budget": 0.73, + "threshold": 0.7532128095626831, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.2513414217361586, + "rescued": 783, + "harmed": 128, + "missed_rescue": 32, + "avoided_harm": 57, + "selected_both_correct": 1012, + "selected_both_wrong": 121 + }, + { + "budget": 0.74, + "threshold": 0.6868958473205566, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.2542721345305262, + "rescued": 789, + "harmed": 130, + "missed_rescue": 26, + "avoided_harm": 55, + "selected_both_correct": 1032, + "selected_both_wrong": 121 + }, + { + "budget": 0.75, + "threshold": 0.5980291962623596, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.2556760692250455, + "rescued": 794, + "harmed": 134, + "missed_rescue": 21, + "avoided_harm": 51, + "selected_both_correct": 1051, + "selected_both_wrong": 121 + }, + { + "budget": 0.76, + "threshold": 0.5109596252441406, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.2590890019728125, + "rescued": 798, + "harmed": 134, + "missed_rescue": 17, + "avoided_harm": 51, + "selected_both_correct": 1075, + "selected_both_wrong": 121 + }, + { + "budget": 0.77, + "threshold": 0.43163156509399414, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.26077884946337493, + "rescued": 802, + "harmed": 138, + "missed_rescue": 13, + "avoided_harm": 47, + "selected_both_correct": 1095, + "selected_both_wrong": 121 + }, + { + "budget": 0.78, + "threshold": 0.25203239917755127, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.2653271612172351, + "rescued": 807, + "harmed": 141, + "missed_rescue": 8, + "avoided_harm": 44, + "selected_both_correct": 1114, + "selected_both_wrong": 122 + }, + { + "budget": 0.79, + "threshold": 0.12126290798187256, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.27195742471443407, + "rescued": 811, + "harmed": 141, + "missed_rescue": 4, + "avoided_harm": 44, + "selected_both_correct": 1138, + "selected_both_wrong": 122 + }, + { + "budget": 0.8, + "threshold": -0.09552910923957825, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.2714259897095718, + "rescued": 812, + "harmed": 143, + "missed_rescue": 3, + "avoided_harm": 42, + "selected_both_correct": 1163, + "selected_both_wrong": 122 + }, + { + "budget": 0.81, + "threshold": -0.1923833191394806, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.27095877937997614, + "rescued": 812, + "harmed": 144, + "missed_rescue": 3, + "avoided_harm": 41, + "selected_both_correct": 1189, + "selected_both_wrong": 123 + }, + { + "budget": 0.82, + "threshold": -0.34329524636268616, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.2702691581790501, + "rescued": 813, + "harmed": 146, + "missed_rescue": 2, + "avoided_harm": 39, + "selected_both_correct": 1214, + "selected_both_wrong": 123 + }, + { + "budget": 0.83, + "threshold": -0.4216182231903076, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.2719739402647948, + "rescued": 814, + "harmed": 149, + "missed_rescue": 1, + "avoided_harm": 36, + "selected_both_correct": 1238, + "selected_both_wrong": 123 + }, + { + "budget": 0.84, + "threshold": -0.4976203441619873, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.27034109882150986, + "rescued": 814, + "harmed": 152, + "missed_rescue": 1, + "avoided_harm": 33, + "selected_both_correct": 1263, + "selected_both_wrong": 123 + }, + { + "budget": 0.85, + "threshold": -0.5450560450553894, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.2707946857828067, + "rescued": 815, + "harmed": 154, + "missed_rescue": 0, + "avoided_harm": 31, + "selected_both_correct": 1288, + "selected_both_wrong": 123 + }, + { + "budget": 0.86, + "threshold": -0.5913483500480652, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.27012633063998975, + "rescued": 815, + "harmed": 155, + "missed_rescue": 0, + "avoided_harm": 30, + "selected_both_correct": 1315, + "selected_both_wrong": 123 + }, + { + "budget": 0.87, + "threshold": -0.6321582794189453, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.2685409698996656, + "rescued": 815, + "harmed": 158, + "missed_rescue": 0, + "avoided_harm": 27, + "selected_both_correct": 1340, + "selected_both_wrong": 123 + }, + { + "budget": 0.88, + "threshold": -0.6923226118087769, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.2669935137326442, + "rescued": 815, + "harmed": 161, + "missed_rescue": 0, + "avoided_harm": 24, + "selected_both_correct": 1365, + "selected_both_wrong": 123 + }, + { + "budget": 0.89, + "threshold": -0.7169911861419678, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.2652998133307999, + "rescued": 815, + "harmed": 164, + "missed_rescue": 0, + "avoided_harm": 21, + "selected_both_correct": 1390, + "selected_both_wrong": 123 + }, + { + "budget": 0.9, + "threshold": -0.7484629154205322, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.26487252377090814, + "rescued": 815, + "harmed": 165, + "missed_rescue": 0, + "avoided_harm": 20, + "selected_both_correct": 1417, + "selected_both_wrong": 123 + }, + { + "budget": 0.91, + "threshold": -0.7902401089668274, + "operator": "ge", + "actual_fraction": 0.9087018544935807, + "metric": 0.2621276637503081, + "rescued": 815, + "harmed": 170, + "missed_rescue": 0, + "avoided_harm": 15, + "selected_both_correct": 1440, + "selected_both_wrong": 123 + }, + { + "budget": 0.92, + "threshold": -0.8233433365821838, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.26132137469568206, + "rescued": 815, + "harmed": 172, + "missed_rescue": 0, + "avoided_harm": 13, + "selected_both_correct": 1466, + "selected_both_wrong": 123 + }, + { + "budget": 0.93, + "threshold": -0.8642669916152954, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.260302736857047, + "rescued": 815, + "harmed": 174, + "missed_rescue": 0, + "avoided_harm": 11, + "selected_both_correct": 1492, + "selected_both_wrong": 123 + }, + { + "budget": 0.94, + "threshold": -0.8947426676750183, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.2595346877019011, + "rescued": 815, + "harmed": 176, + "missed_rescue": 0, + "avoided_harm": 9, + "selected_both_correct": 1518, + "selected_both_wrong": 123 + }, + { + "budget": 0.95, + "threshold": -0.9113625288009644, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.2595346877019011, + "rescued": 815, + "harmed": 176, + "missed_rescue": 0, + "avoided_harm": 9, + "selected_both_correct": 1546, + "selected_both_wrong": 123 + }, + { + "budget": 0.96, + "threshold": -0.9461068511009216, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.2595346877019011, + "rescued": 815, + "harmed": 176, + "missed_rescue": 0, + "avoided_harm": 9, + "selected_both_correct": 1574, + "selected_both_wrong": 123 + }, + { + "budget": 0.97, + "threshold": -1.0160794258117676, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.25915973457104247, + "rescued": 815, + "harmed": 177, + "missed_rescue": 0, + "avoided_harm": 8, + "selected_both_correct": 1601, + "selected_both_wrong": 123 + }, + { + "budget": 0.98, + "threshold": -1.0653282403945923, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.25842726798983023, + "rescued": 815, + "harmed": 179, + "missed_rescue": 0, + "avoided_harm": 6, + "selected_both_correct": 1627, + "selected_both_wrong": 123 + }, + { + "budget": 0.99, + "threshold": -1.1271586418151855, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.25580499348307645, + "rescued": 815, + "harmed": 182, + "missed_rescue": 0, + "avoided_harm": 3, + "selected_both_correct": 1652, + "selected_both_wrong": 123 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.25448626222520154, + "rescued": 815, + "harmed": 185, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 1681, + "selected_both_wrong": 123 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.07887397871043929, + "rescued": 0, + "harmed": 0, + "missed_rescue": 3461, + "avoided_harm": 676, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.754962205886841, + "operator": "ge", + "actual_fraction": 0.013648468708388815, + "metric": 0.07903292594055024, + "rescued": 59, + "harmed": 5, + "missed_rescue": 3402, + "avoided_harm": 671, + "selected_both_correct": 90, + "selected_both_wrong": 10 + }, + { + "budget": 0.02, + "threshold": 2.6711103916168213, + "operator": "ge", + "actual_fraction": 0.018974700399467376, + "metric": 0.07950863651164858, + "rescued": 85, + "harmed": 7, + "missed_rescue": 3376, + "avoided_harm": 669, + "selected_both_correct": 120, + "selected_both_wrong": 16 + }, + { + "budget": 0.03, + "threshold": 2.6189634799957275, + "operator": "ge", + "actual_fraction": 0.021637816245006658, + "metric": 0.07973943420028494, + "rescued": 99, + "harmed": 7, + "missed_rescue": 3362, + "avoided_harm": 669, + "selected_both_correct": 137, + "selected_both_wrong": 17 + }, + { + "budget": 0.04, + "threshold": 2.5298099517822266, + "operator": "ge", + "actual_fraction": 0.030958721704394142, + "metric": 0.08831618109747413, + "rescued": 154, + "harmed": 9, + "missed_rescue": 3307, + "avoided_harm": 667, + "selected_both_correct": 186, + "selected_both_wrong": 23 + }, + { + "budget": 0.05, + "threshold": 2.486783981323242, + "operator": "ge", + "actual_fraction": 0.034287616511318245, + "metric": 0.08842258620588453, + "rescued": 172, + "harmed": 11, + "missed_rescue": 3289, + "avoided_harm": 665, + "selected_both_correct": 202, + "selected_both_wrong": 27 + }, + { + "budget": 0.06, + "threshold": 2.4111104011535645, + "operator": "ge", + "actual_fraction": 0.042276964047936086, + "metric": 0.08900034497323044, + "rescued": 207, + "harmed": 12, + "missed_rescue": 3254, + "avoided_harm": 664, + "selected_both_correct": 254, + "selected_both_wrong": 35 + }, + { + "budget": 0.07, + "threshold": 2.3550784587860107, + "operator": "ge", + "actual_fraction": 0.05326231691078562, + "metric": 0.08981307570330821, + "rescued": 258, + "harmed": 14, + "missed_rescue": 3203, + "avoided_harm": 662, + "selected_both_correct": 322, + "selected_both_wrong": 46 + }, + { + "budget": 0.08, + "threshold": 2.321181297302246, + "operator": "ge", + "actual_fraction": 0.06125166444740346, + "metric": 0.09676973600800028, + "rescued": 299, + "harmed": 17, + "missed_rescue": 3162, + "avoided_harm": 659, + "selected_both_correct": 369, + "selected_both_wrong": 51 + }, + { + "budget": 0.09, + "threshold": 2.26045823097229, + "operator": "ge", + "actual_fraction": 0.0792276964047936, + "metric": 0.09729331364685621, + "rescued": 379, + "harmed": 24, + "missed_rescue": 3082, + "avoided_harm": 652, + "selected_both_correct": 482, + "selected_both_wrong": 67 + }, + { + "budget": 0.1, + "threshold": 2.2216036319732666, + "operator": "ge", + "actual_fraction": 0.09154460719041278, + "metric": 0.09748166643216911, + "rescued": 437, + "harmed": 29, + "missed_rescue": 3024, + "avoided_harm": 647, + "selected_both_correct": 559, + "selected_both_wrong": 75 + }, + { + "budget": 0.11, + "threshold": 2.1819043159484863, + "operator": "ge", + "actual_fraction": 0.10952063914780293, + "metric": 0.10381557967540606, + "rescued": 525, + "harmed": 36, + "missed_rescue": 2936, + "avoided_harm": 640, + "selected_both_correct": 666, + "selected_both_wrong": 89 + }, + { + "budget": 0.12, + "threshold": 2.16599702835083, + "operator": "ge", + "actual_fraction": 0.11351531291611185, + "metric": 0.10430109017434279, + "rescued": 549, + "harmed": 36, + "missed_rescue": 2912, + "avoided_harm": 640, + "selected_both_correct": 690, + "selected_both_wrong": 89 + }, + { + "budget": 0.13, + "threshold": 2.1376547813415527, + "operator": "ge", + "actual_fraction": 0.1248335552596538, + "metric": 0.10450219132130334, + "rescued": 604, + "harmed": 41, + "missed_rescue": 2857, + "avoided_harm": 635, + "selected_both_correct": 752, + "selected_both_wrong": 103 + }, + { + "budget": 0.14, + "threshold": 2.1090269088745117, + "operator": "ge", + "actual_fraction": 0.13448735019973368, + "metric": 0.10490479254367022, + "rescued": 658, + "harmed": 45, + "missed_rescue": 2803, + "avoided_harm": 631, + "selected_both_correct": 804, + "selected_both_wrong": 109 + }, + { + "budget": 0.15, + "threshold": 2.0827982425689697, + "operator": "ge", + "actual_fraction": 0.14747003994673769, + "metric": 0.10538862853749181, + "rescued": 726, + "harmed": 49, + "missed_rescue": 2735, + "avoided_harm": 627, + "selected_both_correct": 882, + "selected_both_wrong": 115 + }, + { + "budget": 0.16, + "threshold": 2.0538668632507324, + "operator": "ge", + "actual_fraction": 0.15878828229027964, + "metric": 0.1098174735753291, + "rescued": 790, + "harmed": 55, + "missed_rescue": 2671, + "avoided_harm": 621, + "selected_both_correct": 942, + "selected_both_wrong": 121 + }, + { + "budget": 0.17, + "threshold": 2.027881145477295, + "operator": "ge", + "actual_fraction": 0.17177097203728361, + "metric": 0.11034871595174692, + "rescued": 846, + "harmed": 61, + "missed_rescue": 2615, + "avoided_harm": 615, + "selected_both_correct": 1022, + "selected_both_wrong": 135 + }, + { + "budget": 0.18, + "threshold": 2.0083131790161133, + "operator": "ge", + "actual_fraction": 0.1807589880159787, + "metric": 0.11470200794701085, + "rescued": 886, + "harmed": 65, + "missed_rescue": 2575, + "avoided_harm": 611, + "selected_both_correct": 1075, + "selected_both_wrong": 146 + }, + { + "budget": 0.19, + "threshold": 1.978614091873169, + "operator": "ge", + "actual_fraction": 0.19207723035952065, + "metric": 0.11937450938005102, + "rescued": 949, + "harmed": 68, + "missed_rescue": 2512, + "avoided_harm": 608, + "selected_both_correct": 1138, + "selected_both_wrong": 153 + }, + { + "budget": 0.2, + "threshold": 1.968393087387085, + "operator": "ge", + "actual_fraction": 0.19540612516644473, + "metric": 0.11970582412121232, + "rescued": 965, + "harmed": 68, + "missed_rescue": 2496, + "avoided_harm": 608, + "selected_both_correct": 1159, + "selected_both_wrong": 156 + }, + { + "budget": 0.21, + "threshold": 1.9526315927505493, + "operator": "ge", + "actual_fraction": 0.20139813581890811, + "metric": 0.1199176042352343, + "rescued": 997, + "harmed": 70, + "missed_rescue": 2464, + "avoided_harm": 606, + "selected_both_correct": 1191, + "selected_both_wrong": 162 + }, + { + "budget": 0.22, + "threshold": 1.936199426651001, + "operator": "ge", + "actual_fraction": 0.21471371504660453, + "metric": 0.12057798910885716, + "rescued": 1057, + "harmed": 78, + "missed_rescue": 2404, + "avoided_harm": 598, + "selected_both_correct": 1266, + "selected_both_wrong": 179 + }, + { + "budget": 0.23, + "threshold": 1.9114916324615479, + "operator": "ge", + "actual_fraction": 0.2303595206391478, + "metric": 0.12102265003769325, + "rescued": 1129, + "harmed": 84, + "missed_rescue": 2332, + "avoided_harm": 592, + "selected_both_correct": 1360, + "selected_both_wrong": 195 + }, + { + "budget": 0.24, + "threshold": 1.9026963710784912, + "operator": "ge", + "actual_fraction": 0.23568575233022637, + "metric": 0.12110388208315034, + "rescued": 1155, + "harmed": 87, + "missed_rescue": 2306, + "avoided_harm": 589, + "selected_both_correct": 1388, + "selected_both_wrong": 202 + }, + { + "budget": 0.25, + "threshold": 1.8903357982635498, + "operator": "ge", + "actual_fraction": 0.24434087882822902, + "metric": 0.12179158603990739, + "rescued": 1196, + "harmed": 89, + "missed_rescue": 2265, + "avoided_harm": 587, + "selected_both_correct": 1441, + "selected_both_wrong": 210 + }, + { + "budget": 0.26, + "threshold": 1.8735990524291992, + "operator": "ge", + "actual_fraction": 0.2569906790945406, + "metric": 0.12300356188591355, + "rescued": 1254, + "harmed": 91, + "missed_rescue": 2207, + "avoided_harm": 585, + "selected_both_correct": 1523, + "selected_both_wrong": 220 + }, + { + "budget": 0.27, + "threshold": 1.8593947887420654, + "operator": "ge", + "actual_fraction": 0.2673102529960053, + "metric": 0.12289074126954411, + "rescued": 1297, + "harmed": 99, + "missed_rescue": 2164, + "avoided_harm": 577, + "selected_both_correct": 1588, + "selected_both_wrong": 228 + }, + { + "budget": 0.28, + "threshold": 1.841538429260254, + "operator": "ge", + "actual_fraction": 0.2802929427430093, + "metric": 0.12433964952036403, + "rescued": 1361, + "harmed": 113, + "missed_rescue": 2100, + "avoided_harm": 563, + "selected_both_correct": 1655, + "selected_both_wrong": 239 + }, + { + "budget": 0.29, + "threshold": 1.8348605632781982, + "operator": "ge", + "actual_fraction": 0.28395472703062585, + "metric": 0.12431245080773455, + "rescued": 1375, + "harmed": 116, + "missed_rescue": 2086, + "avoided_harm": 560, + "selected_both_correct": 1680, + "selected_both_wrong": 241 + }, + { + "budget": 0.3, + "threshold": 1.8283830881118774, + "operator": "ge", + "actual_fraction": 0.28894806924101196, + "metric": 0.12479938831348537, + "rescued": 1401, + "harmed": 118, + "missed_rescue": 2060, + "avoided_harm": 558, + "selected_both_correct": 1709, + "selected_both_wrong": 244 + }, + { + "budget": 0.31, + "threshold": 1.8149042129516602, + "operator": "ge", + "actual_fraction": 0.29760319573901467, + "metric": 0.13146961100935234, + "rescued": 1441, + "harmed": 123, + "missed_rescue": 2020, + "avoided_harm": 553, + "selected_both_correct": 1760, + "selected_both_wrong": 252 + }, + { + "budget": 0.32, + "threshold": 1.8025038242340088, + "operator": "ge", + "actual_fraction": 0.30758988015978694, + "metric": 0.1322835989872101, + "rescued": 1480, + "harmed": 136, + "missed_rescue": 1981, + "avoided_harm": 540, + "selected_both_correct": 1822, + "selected_both_wrong": 258 + }, + { + "budget": 0.33, + "threshold": 1.7916626930236816, + "operator": "ge", + "actual_fraction": 0.31657789613848203, + "metric": 0.1328527201611299, + "rescued": 1522, + "harmed": 140, + "missed_rescue": 1939, + "avoided_harm": 536, + "selected_both_correct": 1873, + "selected_both_wrong": 269 + }, + { + "budget": 0.34, + "threshold": 1.7744719982147217, + "operator": "ge", + "actual_fraction": 0.3292276964047936, + "metric": 0.13322103039680225, + "rescued": 1574, + "harmed": 148, + "missed_rescue": 1887, + "avoided_harm": 528, + "selected_both_correct": 1952, + "selected_both_wrong": 282 + }, + { + "budget": 0.35, + "threshold": 1.7519283294677734, + "operator": "ge", + "actual_fraction": 0.3445406125166445, + "metric": 0.13642917552598965, + "rescued": 1654, + "harmed": 159, + "missed_rescue": 1807, + "avoided_harm": 517, + "selected_both_correct": 2036, + "selected_both_wrong": 291 + }, + { + "budget": 0.36, + "threshold": 1.743757963180542, + "operator": "ge", + "actual_fraction": 0.3492010652463382, + "metric": 0.13678368923440343, + "rescued": 1673, + "harmed": 160, + "missed_rescue": 1788, + "avoided_harm": 516, + "selected_both_correct": 2067, + "selected_both_wrong": 296 + }, + { + "budget": 0.37, + "threshold": 1.7330536842346191, + "operator": "ge", + "actual_fraction": 0.3558588548601864, + "metric": 0.13783347849529223, + "rescued": 1707, + "harmed": 160, + "missed_rescue": 1754, + "avoided_harm": 516, + "selected_both_correct": 2108, + "selected_both_wrong": 301 + }, + { + "budget": 0.38, + "threshold": 1.7175750732421875, + "operator": "ge", + "actual_fraction": 0.36351531291611183, + "metric": 0.14054378238413273, + "rescued": 1749, + "harmed": 168, + "missed_rescue": 1712, + "avoided_harm": 508, + "selected_both_correct": 2149, + "selected_both_wrong": 302 + }, + { + "budget": 0.39, + "threshold": 1.7006492614746094, + "operator": "ge", + "actual_fraction": 0.3745006657789614, + "metric": 0.14225192159448571, + "rescued": 1800, + "harmed": 170, + "missed_rescue": 1661, + "avoided_harm": 506, + "selected_both_correct": 2224, + "selected_both_wrong": 306 + }, + { + "budget": 0.4, + "threshold": 1.6880998611450195, + "operator": "ge", + "actual_fraction": 0.38049267643142476, + "metric": 0.1426740526941028, + "rescued": 1828, + "harmed": 174, + "missed_rescue": 1633, + "avoided_harm": 502, + "selected_both_correct": 2259, + "selected_both_wrong": 311 + }, + { + "budget": 0.41, + "threshold": 1.6769726276397705, + "operator": "ge", + "actual_fraction": 0.3914780292942743, + "metric": 0.14443415768811052, + "rescued": 1880, + "harmed": 184, + "missed_rescue": 1581, + "avoided_harm": 492, + "selected_both_correct": 2322, + "selected_both_wrong": 318 + }, + { + "budget": 0.42, + "threshold": 1.647587537765503, + "operator": "ge", + "actual_fraction": 0.4111185086551265, + "metric": 0.14610051676403063, + "rescued": 1969, + "harmed": 198, + "missed_rescue": 1492, + "avoided_harm": 478, + "selected_both_correct": 2438, + "selected_both_wrong": 335 + }, + { + "budget": 0.43, + "threshold": 1.6429295539855957, + "operator": "ge", + "actual_fraction": 0.41544607190412786, + "metric": 0.14685373029661633, + "rescued": 1989, + "harmed": 202, + "missed_rescue": 1472, + "avoided_harm": 474, + "selected_both_correct": 2463, + "selected_both_wrong": 338 + }, + { + "budget": 0.44, + "threshold": 1.6297134160995483, + "operator": "ge", + "actual_fraction": 0.42243675099866845, + "metric": 0.14696422698695188, + "rescued": 2019, + "harmed": 209, + "missed_rescue": 1442, + "avoided_harm": 467, + "selected_both_correct": 2506, + "selected_both_wrong": 342 + }, + { + "budget": 0.45, + "threshold": 1.6237554550170898, + "operator": "ge", + "actual_fraction": 0.4264314247669774, + "metric": 0.14731149063585433, + "rescued": 2034, + "harmed": 210, + "missed_rescue": 1427, + "avoided_harm": 466, + "selected_both_correct": 2535, + "selected_both_wrong": 345 + }, + { + "budget": 0.46, + "threshold": 1.6143747568130493, + "operator": "ge", + "actual_fraction": 0.4337549933422104, + "metric": 0.15035049533448158, + "rescued": 2071, + "harmed": 215, + "missed_rescue": 1390, + "avoided_harm": 461, + "selected_both_correct": 2577, + "selected_both_wrong": 349 + }, + { + "budget": 0.47, + "threshold": 1.5888383388519287, + "operator": "ge", + "actual_fraction": 0.4503994673768309, + "metric": 0.15138848539889838, + "rescued": 2130, + "harmed": 229, + "missed_rescue": 1331, + "avoided_harm": 447, + "selected_both_correct": 2690, + "selected_both_wrong": 363 + }, + { + "budget": 0.48, + "threshold": 1.5711208581924438, + "operator": "ge", + "actual_fraction": 0.46171770972037285, + "metric": 0.1542600136742569, + "rescued": 2183, + "harmed": 239, + "missed_rescue": 1278, + "avoided_harm": 437, + "selected_both_correct": 2752, + "selected_both_wrong": 374 + }, + { + "budget": 0.49, + "threshold": 1.5494966506958008, + "operator": "ge", + "actual_fraction": 0.4780292942743009, + "metric": 0.16026067412653672, + "rescued": 2250, + "harmed": 248, + "missed_rescue": 1211, + "avoided_harm": 428, + "selected_both_correct": 2862, + "selected_both_wrong": 384 + }, + { + "budget": 0.5, + "threshold": 1.5336257219314575, + "operator": "ge", + "actual_fraction": 0.48601864181091875, + "metric": 0.16374337431326194, + "rescued": 2284, + "harmed": 249, + "missed_rescue": 1177, + "avoided_harm": 427, + "selected_both_correct": 2919, + "selected_both_wrong": 388 + }, + { + "budget": 0.51, + "threshold": 1.5163730382919312, + "operator": "ge", + "actual_fraction": 0.4986684420772304, + "metric": 0.16649880286803048, + "rescued": 2335, + "harmed": 266, + "missed_rescue": 1126, + "avoided_harm": 410, + "selected_both_correct": 2991, + "selected_both_wrong": 400 + }, + { + "budget": 0.52, + "threshold": 1.5053234100341797, + "operator": "ge", + "actual_fraction": 0.501664447403462, + "metric": 0.16716166299031399, + "rescued": 2350, + "harmed": 267, + "missed_rescue": 1111, + "avoided_harm": 409, + "selected_both_correct": 3008, + "selected_both_wrong": 403 + }, + { + "budget": 0.53, + "threshold": 1.4833742380142212, + "operator": "ge", + "actual_fraction": 0.5186418109187749, + "metric": 0.17701801911105697, + "rescued": 2429, + "harmed": 275, + "missed_rescue": 1032, + "avoided_harm": 401, + "selected_both_correct": 3111, + "selected_both_wrong": 417 + }, + { + "budget": 0.54, + "threshold": 1.4717422723770142, + "operator": "ge", + "actual_fraction": 0.5246338215712384, + "metric": 0.17996234745125575, + "rescued": 2460, + "harmed": 279, + "missed_rescue": 1001, + "avoided_harm": 397, + "selected_both_correct": 3146, + "selected_both_wrong": 419 + }, + { + "budget": 0.55, + "threshold": 1.4611992835998535, + "operator": "ge", + "actual_fraction": 0.5309587217043942, + "metric": 0.1806533171837632, + "rescued": 2484, + "harmed": 284, + "missed_rescue": 977, + "avoided_harm": 392, + "selected_both_correct": 3188, + "selected_both_wrong": 424 + }, + { + "budget": 0.56, + "threshold": 1.444828748703003, + "operator": "ge", + "actual_fraction": 0.5426098535286284, + "metric": 0.18341049095176862, + "rescued": 2528, + "harmed": 293, + "missed_rescue": 933, + "avoided_harm": 383, + "selected_both_correct": 3264, + "selected_both_wrong": 435 + }, + { + "budget": 0.57, + "threshold": 1.4202982187271118, + "operator": "ge", + "actual_fraction": 0.5565912117177098, + "metric": 0.18685925293119887, + "rescued": 2590, + "harmed": 307, + "missed_rescue": 871, + "avoided_harm": 369, + "selected_both_correct": 3342, + "selected_both_wrong": 449 + }, + { + "budget": 0.58, + "threshold": 1.402458667755127, + "operator": "ge", + "actual_fraction": 0.5659121171770972, + "metric": 0.18915205854128653, + "rescued": 2627, + "harmed": 314, + "missed_rescue": 834, + "avoided_harm": 362, + "selected_both_correct": 3405, + "selected_both_wrong": 454 + }, + { + "budget": 0.59, + "threshold": 1.3859935998916626, + "operator": "ge", + "actual_fraction": 0.5765645805592543, + "metric": 0.1905802005819197, + "rescued": 2671, + "harmed": 325, + "missed_rescue": 790, + "avoided_harm": 351, + "selected_both_correct": 3472, + "selected_both_wrong": 460 + }, + { + "budget": 0.6, + "threshold": 1.3550105094909668, + "operator": "ge", + "actual_fraction": 0.5915446071904128, + "metric": 0.19574045934947382, + "rescued": 2730, + "harmed": 340, + "missed_rescue": 731, + "avoided_harm": 336, + "selected_both_correct": 3569, + "selected_both_wrong": 469 + }, + { + "budget": 0.61, + "threshold": 1.336022973060608, + "operator": "ge", + "actual_fraction": 0.6005326231691078, + "metric": 0.19897575440094883, + "rescued": 2769, + "harmed": 348, + "missed_rescue": 692, + "avoided_harm": 328, + "selected_both_correct": 3626, + "selected_both_wrong": 473 + }, + { + "budget": 0.62, + "threshold": 1.2876667976379395, + "operator": "ge", + "actual_fraction": 0.6188415446071904, + "metric": 0.20144209535288454, + "rescued": 2849, + "harmed": 367, + "missed_rescue": 612, + "avoided_harm": 309, + "selected_both_correct": 3737, + "selected_both_wrong": 483 + }, + { + "budget": 0.63, + "threshold": 1.2704218626022339, + "operator": "ge", + "actual_fraction": 0.6278295605858855, + "metric": 0.20418490548524335, + "rescued": 2889, + "harmed": 379, + "missed_rescue": 572, + "avoided_harm": 297, + "selected_both_correct": 3786, + "selected_both_wrong": 490 + }, + { + "budget": 0.64, + "threshold": 1.2135732173919678, + "operator": "ge", + "actual_fraction": 0.6461384820239681, + "metric": 0.20927276648728108, + "rescued": 2967, + "harmed": 396, + "missed_rescue": 494, + "avoided_harm": 280, + "selected_both_correct": 3900, + "selected_both_wrong": 501 + }, + { + "budget": 0.65, + "threshold": 1.1857013702392578, + "operator": "ge", + "actual_fraction": 0.6537949400798935, + "metric": 0.21139933529545984, + "rescued": 2995, + "harmed": 403, + "missed_rescue": 466, + "avoided_harm": 273, + "selected_both_correct": 3955, + "selected_both_wrong": 503 + }, + { + "budget": 0.66, + "threshold": 1.1139411926269531, + "operator": "ge", + "actual_fraction": 0.6764314247669774, + "metric": 0.22236862909896596, + "rescued": 3089, + "harmed": 428, + "missed_rescue": 372, + "avoided_harm": 248, + "selected_both_correct": 4097, + "selected_both_wrong": 514 + }, + { + "budget": 0.67, + "threshold": 1.0512058734893799, + "operator": "ge", + "actual_fraction": 0.6947403462050599, + "metric": 0.23100465175745166, + "rescued": 3166, + "harmed": 448, + "missed_rescue": 295, + "avoided_harm": 228, + "selected_both_correct": 4212, + "selected_both_wrong": 522 + }, + { + "budget": 0.68, + "threshold": 1.0203181505203247, + "operator": "ge", + "actual_fraction": 0.7033954727030626, + "metric": 0.23344075252006277, + "rescued": 3193, + "harmed": 458, + "missed_rescue": 268, + "avoided_harm": 218, + "selected_both_correct": 4276, + "selected_both_wrong": 525 + }, + { + "budget": 0.69, + "threshold": 0.980606198310852, + "operator": "ge", + "actual_fraction": 0.7137150466045273, + "metric": 0.23922820682021304, + "rescued": 3235, + "harmed": 472, + "missed_rescue": 226, + "avoided_harm": 204, + "selected_both_correct": 4341, + "selected_both_wrong": 528 + }, + { + "budget": 0.7, + "threshold": 0.9318755269050598, + "operator": "ge", + "actual_fraction": 0.7197070572569907, + "metric": 0.24161777755845307, + "rescued": 3256, + "harmed": 481, + "missed_rescue": 205, + "avoided_harm": 195, + "selected_both_correct": 4383, + "selected_both_wrong": 528 + }, + { + "budget": 0.71, + "threshold": 0.854070782661438, + "operator": "ge", + "actual_fraction": 0.7290279627163782, + "metric": 0.24493479982060765, + "rescued": 3284, + "harmed": 488, + "missed_rescue": 177, + "avoided_harm": 188, + "selected_both_correct": 4457, + "selected_both_wrong": 531 + }, + { + "budget": 0.72, + "threshold": 0.8128365278244019, + "operator": "ge", + "actual_fraction": 0.7346870838881492, + "metric": 0.24679495225212467, + "rescued": 3299, + "harmed": 492, + "missed_rescue": 162, + "avoided_harm": 184, + "selected_both_correct": 4505, + "selected_both_wrong": 532 + }, + { + "budget": 0.73, + "threshold": 0.7532128095626831, + "operator": "ge", + "actual_fraction": 0.7396804260985352, + "metric": 0.24966549891509135, + "rescued": 3315, + "harmed": 498, + "missed_rescue": 146, + "avoided_harm": 178, + "selected_both_correct": 4543, + "selected_both_wrong": 532 + }, + { + "budget": 0.74, + "threshold": 0.6868958473205566, + "operator": "ge", + "actual_fraction": 0.7480026631158455, + "metric": 0.2538474002928039, + "rescued": 3334, + "harmed": 506, + "missed_rescue": 127, + "avoided_harm": 170, + "selected_both_correct": 4610, + "selected_both_wrong": 538 + }, + { + "budget": 0.75, + "threshold": 0.5980291962623596, + "operator": "ge", + "actual_fraction": 0.7563249001331558, + "metric": 0.25876835252498565, + "rescued": 3358, + "harmed": 510, + "missed_rescue": 103, + "avoided_harm": 166, + "selected_both_correct": 4679, + "selected_both_wrong": 541 + }, + { + "budget": 0.76, + "threshold": 0.5109596252441406, + "operator": "ge", + "actual_fraction": 0.7626498002663116, + "metric": 0.26093598369162163, + "rescued": 3374, + "harmed": 517, + "missed_rescue": 87, + "avoided_harm": 159, + "selected_both_correct": 4729, + "selected_both_wrong": 544 + }, + { + "budget": 0.77, + "threshold": 0.43163156509399414, + "operator": "ge", + "actual_fraction": 0.7686418109187749, + "metric": 0.2625072008694927, + "rescued": 3387, + "harmed": 522, + "missed_rescue": 74, + "avoided_harm": 154, + "selected_both_correct": 4782, + "selected_both_wrong": 545 + }, + { + "budget": 0.78, + "threshold": 0.25203239917755127, + "operator": "ge", + "actual_fraction": 0.7809587217043942, + "metric": 0.2664938664875827, + "rescued": 3414, + "harmed": 534, + "missed_rescue": 47, + "avoided_harm": 142, + "selected_both_correct": 4890, + "selected_both_wrong": 546 + }, + { + "budget": 0.79, + "threshold": 0.12126290798187256, + "operator": "ge", + "actual_fraction": 0.7869507323568575, + "metric": 0.26849228241257794, + "rescued": 3426, + "harmed": 539, + "missed_rescue": 35, + "avoided_harm": 137, + "selected_both_correct": 4944, + "selected_both_wrong": 547 + }, + { + "budget": 0.8, + "threshold": -0.09552910923957825, + "operator": "ge", + "actual_fraction": 0.8012649800266312, + "metric": 0.2686957195815463, + "rescued": 3443, + "harmed": 557, + "missed_rescue": 18, + "avoided_harm": 119, + "selected_both_correct": 5081, + "selected_both_wrong": 547 + }, + { + "budget": 0.81, + "threshold": -0.1923833191394806, + "operator": "ge", + "actual_fraction": 0.8055925432756325, + "metric": 0.26896300567216713, + "rescued": 3446, + "harmed": 560, + "missed_rescue": 15, + "avoided_harm": 116, + "selected_both_correct": 5127, + "selected_both_wrong": 547 + }, + { + "budget": 0.82, + "threshold": -0.34329524636268616, + "operator": "ge", + "actual_fraction": 0.8185752330226365, + "metric": 0.26854198500532145, + "rescued": 3448, + "harmed": 567, + "missed_rescue": 13, + "avoided_harm": 109, + "selected_both_correct": 5273, + "selected_both_wrong": 548 + }, + { + "budget": 0.83, + "threshold": -0.4216182231903076, + "operator": "ge", + "actual_fraction": 0.831225033288948, + "metric": 0.26693203615113315, + "rescued": 3448, + "harmed": 576, + "missed_rescue": 13, + "avoided_harm": 100, + "selected_both_correct": 5415, + "selected_both_wrong": 549 + }, + { + "budget": 0.84, + "threshold": -0.4976203441619873, + "operator": "ge", + "actual_fraction": 0.8445406125166445, + "metric": 0.2678966254593594, + "rescued": 3450, + "harmed": 582, + "missed_rescue": 11, + "avoided_harm": 94, + "selected_both_correct": 5567, + "selected_both_wrong": 549 + }, + { + "budget": 0.85, + "threshold": -0.5450560450553894, + "operator": "ge", + "actual_fraction": 0.8528628495339547, + "metric": 0.2674776330506041, + "rescued": 3450, + "harmed": 585, + "missed_rescue": 11, + "avoided_harm": 91, + "selected_both_correct": 5663, + "selected_both_wrong": 550 + }, + { + "budget": 0.86, + "threshold": -0.5913483500480652, + "operator": "ge", + "actual_fraction": 0.8615179760319573, + "metric": 0.26685434544589476, + "rescued": 3450, + "harmed": 591, + "missed_rescue": 11, + "avoided_harm": 85, + "selected_both_correct": 5761, + "selected_both_wrong": 550 + }, + { + "budget": 0.87, + "threshold": -0.6321582794189453, + "operator": "ge", + "actual_fraction": 0.8705059920106525, + "metric": 0.2661351461543834, + "rescued": 3450, + "harmed": 596, + "missed_rescue": 11, + "avoided_harm": 80, + "selected_both_correct": 5864, + "selected_both_wrong": 550 + }, + { + "budget": 0.88, + "threshold": -0.6923226118087769, + "operator": "ge", + "actual_fraction": 0.8841544607190412, + "metric": 0.266263626593725, + "rescued": 3452, + "harmed": 601, + "missed_rescue": 9, + "avoided_harm": 75, + "selected_both_correct": 6021, + "selected_both_wrong": 550 + }, + { + "budget": 0.89, + "threshold": -0.7169911861419678, + "operator": "ge", + "actual_fraction": 0.8908122503328895, + "metric": 0.26592568170237746, + "rescued": 3452, + "harmed": 603, + "missed_rescue": 9, + "avoided_harm": 73, + "selected_both_correct": 6099, + "selected_both_wrong": 550 + }, + { + "budget": 0.9, + "threshold": -0.7484629154205322, + "operator": "ge", + "actual_fraction": 0.8994673768308922, + "metric": 0.26554719122260856, + "rescued": 3452, + "harmed": 607, + "missed_rescue": 9, + "avoided_harm": 69, + "selected_both_correct": 6199, + "selected_both_wrong": 550 + }, + { + "budget": 0.91, + "threshold": -0.7902401089668274, + "operator": "ge", + "actual_fraction": 0.9111185086551264, + "metric": 0.26820679793479935, + "rescued": 3455, + "harmed": 615, + "missed_rescue": 6, + "avoided_harm": 61, + "selected_both_correct": 6328, + "selected_both_wrong": 550 + }, + { + "budget": 0.92, + "threshold": -0.8233433365821838, + "operator": "ge", + "actual_fraction": 0.9221038615179761, + "metric": 0.26798873241822635, + "rescued": 3455, + "harmed": 618, + "missed_rescue": 6, + "avoided_harm": 58, + "selected_both_correct": 6457, + "selected_both_wrong": 550 + }, + { + "budget": 0.93, + "threshold": -0.8642669916152954, + "operator": "ge", + "actual_fraction": 0.9320905459387483, + "metric": 0.26804958873007556, + "rescued": 3456, + "harmed": 621, + "missed_rescue": 5, + "avoided_harm": 55, + "selected_both_correct": 6573, + "selected_both_wrong": 550 + }, + { + "budget": 0.94, + "threshold": -0.8947426676750183, + "operator": "ge", + "actual_fraction": 0.9397470039946738, + "metric": 0.2689187642465245, + "rescued": 3457, + "harmed": 631, + "missed_rescue": 4, + "avoided_harm": 45, + "selected_both_correct": 6654, + "selected_both_wrong": 550 + }, + { + "budget": 0.95, + "threshold": -0.9113625288009644, + "operator": "ge", + "actual_fraction": 0.9437416777629827, + "metric": 0.26868921311665, + "rescued": 3457, + "harmed": 633, + "missed_rescue": 4, + "avoided_harm": 43, + "selected_both_correct": 6700, + "selected_both_wrong": 550 + }, + { + "budget": 0.96, + "threshold": -0.9461068511009216, + "operator": "ge", + "actual_fraction": 0.9510652463382158, + "metric": 0.2681624316438305, + "rescued": 3457, + "harmed": 638, + "missed_rescue": 4, + "avoided_harm": 38, + "selected_both_correct": 6783, + "selected_both_wrong": 550 + }, + { + "budget": 0.97, + "threshold": -1.0160794258117676, + "operator": "ge", + "actual_fraction": 0.9697070572569907, + "metric": 0.2697042218335046, + "rescued": 3459, + "harmed": 649, + "missed_rescue": 2, + "avoided_harm": 27, + "selected_both_correct": 6994, + "selected_both_wrong": 550 + }, + { + "budget": 0.98, + "threshold": -1.0653282403945923, + "operator": "ge", + "actual_fraction": 0.9800266311584553, + "metric": 0.2690008459987387, + "rescued": 3459, + "harmed": 654, + "missed_rescue": 2, + "avoided_harm": 22, + "selected_both_correct": 7113, + "selected_both_wrong": 550 + }, + { + "budget": 0.99, + "threshold": -1.1271586418151855, + "operator": "ge", + "actual_fraction": 0.9860186418109188, + "metric": 0.26914685339444727, + "rescued": 3460, + "harmed": 660, + "missed_rescue": 1, + "avoided_harm": 16, + "selected_both_correct": 7178, + "selected_both_wrong": 550 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.27001009067886245, + "rescued": 3461, + "harmed": 676, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 7329, + "selected_both_wrong": 550 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 1681, + "rescuable": 815, + "harmful": 185, + "both_wrong": 123, + "unit": "event", + "total": 2804 + }, + "test": { + "both_correct": 7329, + "rescuable": 3461, + "harmful": 676, + "both_wrong": 550, + "unit": "event", + "total": 12016 + } + }, + "small": { + "validation": 0.07641657905799261, + "test": 0.07887397871043929 + }, + "large": { + "validation": 0.25448626222520154, + "test": 0.27001009067886245 + }, + "learned_minus_confidence_test": -0.001252276683998299, + "learned_minus_small_test": 0.18805805744069387, + "learned_minus_large_test": -0.0030780545277293014 + } + ], + "summary": { + "learned_beats_confidence": 11, + "learned_beats_both": 5, + "learned_beats_large": 10 + } +} \ No newline at end of file diff --git a/paper_results/data/scaling_selected.json b/paper_results/data/scaling_selected.json new file mode 100644 index 0000000000000000000000000000000000000000..13b8eed7d75aa48de23cc24acd760a2611b272e4 --- /dev/null +++ b/paper_results/data/scaling_selected.json @@ -0,0 +1,1250 @@ +[ + { + "key": [ + "cub", + "cub", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0185", + "tag": "baseline_25pct", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7849499482222989, + "fixed25_actual_fraction": 0.2587159130134622, + "fixed25_rescue": 31, + "fixed25_harm": 201, + "rank25_metric": 0.7884017949603037, + "validation_selection": 0.9741666666666666, + "best_epoch": 12, + "seconds": 5.802937984466553 + }, + { + "id": "formal_0186", + "tag": "baseline_50pct", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7904729030031067, + "fixed25_actual_fraction": 0.2549188816016569, + "fixed25_rescue": 38, + "fixed25_harm": 176, + "rank25_metric": 0.7930617880566103, + "validation_selection": 0.9731666666666667, + "best_epoch": 7, + "seconds": 7.1614015102386475 + }, + { + "id": "formal_0001", + "tag": "baseline_100pct", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.787193648602002, + "fixed25_actual_fraction": 0.2557818432861581, + "fixed25_rescue": 33, + "fixed25_harm": 190, + "rank25_metric": 0.7913358646876079, + "validation_selection": 0.9744166666666666, + "best_epoch": 7, + "seconds": 2.9200518131256104 + } + ] + }, + { + "key": [ + "cub", + "cub", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0187", + "tag": "baseline_25pct", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6874352778736624, + "fixed25_actual_fraction": 0.6039005868139454, + "fixed25_rescue": 358, + "fixed25_harm": 1093, + "rank25_metric": 0.8060062133241284, + "validation_selection": 0.9316666666666666, + "best_epoch": 5, + "seconds": 4.623678922653198 + }, + { + "id": "formal_0188", + "tag": "baseline_50pct", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.694338971349672, + "fixed25_actual_fraction": 0.5997583707283397, + "fixed25_rescue": 357, + "fixed25_harm": 1052, + "rank25_metric": 0.8096306523990335, + "validation_selection": 0.9334166666666667, + "best_epoch": 3, + "seconds": 4.7386627197265625 + }, + { + "id": "formal_0009", + "tag": "baseline_100pct", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6829478771142561, + "fixed25_actual_fraction": 0.6049361408353469, + "fixed25_rescue": 358, + "fixed25_harm": 1119, + "rank25_metric": 0.8091128753883328, + "validation_selection": 0.928, + "best_epoch": 1, + "seconds": 2.053396463394165 + } + ] + }, + { + "key": [ + "cub", + "glsim", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0189", + "tag": "baseline_25pct", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8674490852606145, + "fixed25_actual_fraction": 0.24818778046254747, + "fixed25_rescue": 11, + "fixed25_harm": 209, + "rank25_metric": 0.8674490852606145, + "validation_selection": 0.9786666666666667, + "best_epoch": 12, + "seconds": 6.889333963394165 + }, + { + "id": "formal_0190", + "tag": "baseline_50pct", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.870037970314118, + "fixed25_actual_fraction": 0.24991370383154987, + "fixed25_rescue": 13, + "fixed25_harm": 196, + "rank25_metric": 0.870037970314118, + "validation_selection": 0.9773333333333334, + "best_epoch": 6, + "seconds": 5.409059524536133 + }, + { + "id": "formal_0019", + "tag": "baseline_100pct", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8658957542285123, + "fixed25_actual_fraction": 0.25509147393855713, + "fixed25_rescue": 13, + "fixed25_harm": 220, + "rank25_metric": 0.8679668622713151, + "validation_selection": 0.9788333333333333, + "best_epoch": 9, + "seconds": 7.6912243366241455 + } + ] + }, + { + "key": [ + "cub", + "glsim", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0191", + "tag": "baseline_25pct", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8797031411805316, + "fixed25_actual_fraction": 0.23662409389023129, + "fixed25_rescue": 18, + "fixed25_harm": 145, + "rank25_metric": 0.8776320331377286, + "validation_selection": 0.98025, + "best_epoch": 18, + "seconds": 7.610384702682495 + }, + { + "id": "formal_0192", + "tag": "baseline_50pct", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.876423886779427, + "fixed25_actual_fraction": 0.24525371073524335, + "fixed25_rescue": 8, + "fixed25_harm": 154, + "rank25_metric": 0.8762512944425268, + "validation_selection": 0.981, + "best_epoch": 25, + "seconds": 10.510464906692505 + }, + { + "id": "formal_0027", + "tag": "baseline_100pct", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8684846392820159, + "fixed25_actual_fraction": 0.2507766655160511, + "fixed25_rescue": 17, + "fixed25_harm": 209, + "rank25_metric": 0.8684846392820159, + "validation_selection": 0.97525, + "best_epoch": 1, + "seconds": 3.935011863708496 + } + ] + }, + { + "key": [ + "grefcoco", + "groundingdino", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0193", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.38158976831239544, + "fixed25_actual_fraction": 0.24836230609987806, + "fixed25_rescue": 4083, + "fixed25_harm": 38, + "rank25_metric": 0.38201514335138814, + "validation_selection": 0.6237472766884531, + "best_epoch": 1, + "seconds": 28.353870153427124 + }, + { + "id": "formal_0194", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3658792502055979, + "fixed25_actual_fraction": 0.22034427019822478, + "fixed25_rescue": 3503, + "fixed25_harm": 12, + "rank25_metric": 0.3758330261180274, + "validation_selection": 0.6238526951999438, + "best_epoch": 2, + "seconds": 56.78019833564758 + }, + { + "id": "formal_0042", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3693106088534725, + "fixed25_actual_fraction": 0.22853982928281769, + "fixed25_rescue": 3642, + "fixed25_harm": 30, + "rank25_metric": 0.3758330261180274, + "validation_selection": 0.6221730269168599, + "best_epoch": 9, + "seconds": 476.6986792087555 + } + ] + }, + { + "key": [ + "grefcoco", + "groundingdino", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0195", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4562572668235828, + "fixed25_actual_fraction": 0.24856081445140799, + "fixed25_rescue": 6732, + "fixed25_harm": 54, + "rank25_metric": 0.45705130022970253, + "validation_selection": 0.6656265373532926, + "best_epoch": 1, + "seconds": 18.048375368118286 + }, + { + "id": "formal_0196", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.434648214842753, + "fixed25_actual_fraction": 0.22008904517482913, + "fixed25_rescue": 5930, + "fixed25_harm": 14, + "rank25_metric": 0.4536766582536937, + "validation_selection": 0.6660130718954248, + "best_epoch": 2, + "seconds": 33.53883171081543 + }, + { + "id": "formal_0057", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.43033774778096023, + "fixed25_actual_fraction": 0.2126591611604231, + "fixed25_rescue": 5784, + "fixed25_harm": 20, + "rank25_metric": 0.4544139749879477, + "validation_selection": 0.6650924168950734, + "best_epoch": 5, + "seconds": 281.1101198196411 + } + ] + }, + { + "key": [ + "grefcoco", + "instancevg", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0197", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6523268014632901, + "fixed25_actual_fraction": 0.3479284235601055, + "fixed25_rescue": 1394, + "fixed25_harm": 2000, + "rank25_metric": 0.6652582026486685, + "validation_selection": 0.753925082577834, + "best_epoch": 13, + "seconds": 28.585034132003784 + }, + { + "id": "formal_0198", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6567223435328815, + "fixed25_actual_fraction": 0.3123670703003148, + "fixed25_rescue": 1366, + "fixed25_harm": 1817, + "rank25_metric": 0.662762669086578, + "validation_selection": 0.7576779815868999, + "best_epoch": 13, + "seconds": 44.9037389755249 + }, + { + "id": "formal_0072", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.654765618353515, + "fixed25_actual_fraction": 0.33953435612398264, + "fixed25_rescue": 1510, + "fixed25_harm": 2030, + "rank25_metric": 0.6655985026798628, + "validation_selection": 0.7589219200224893, + "best_epoch": 3, + "seconds": 163.15981340408325 + } + ] + }, + { + "key": [ + "grefcoco", + "instancevg", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0199", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7019538893457732, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 14.837297677993774 + }, + { + "id": "formal_0200", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7416555596517597, + "fixed25_actual_fraction": 0.4823752942177353, + "fixed25_rescue": 3418, + "fixed25_harm": 874, + "rank25_metric": 0.7207838244051838, + "validation_selection": 0.8144985592803429, + "best_epoch": 1, + "seconds": 21.554462432861328 + }, + { + "id": "formal_0083", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7090151149930523, + "fixed25_actual_fraction": 0.12330204463602076, + "fixed25_rescue": 1514, + "fixed25_harm": 121, + "rank25_metric": 0.7021523976973031, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 121.36263680458069 + } + ] + }, + { + "key": [ + "nlvr2", + "nlvr2", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0201", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.863815742268781, + "fixed25_actual_fraction": 0.253928392049939, + "fixed25_rescue": 1100, + "fixed25_harm": 211, + "rank25_metric": 0.8630264762861448, + "validation_selection": 0.8331853337152678, + "best_epoch": 2, + "seconds": 11.065314054489136 + }, + { + "id": "formal_0202", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.868336083805697, + "fixed25_actual_fraction": 0.24388318863456984, + "fixed25_rescue": 1152, + "fixed25_harm": 200, + "rank25_metric": 0.8689818468823993, + "validation_selection": 0.8330850759094816, + "best_epoch": 3, + "seconds": 16.297266960144043 + }, + { + "id": "formal_0091", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8707756332065725, + "fixed25_actual_fraction": 0.26117528879959817, + "fixed25_rescue": 1197, + "fixed25_harm": 211, + "rank25_metric": 0.8689100954294324, + "validation_selection": 0.8335577198510455, + "best_epoch": 6, + "seconds": 82.51407408714294 + } + ] + }, + { + "key": [ + "nlvr2", + "nlvr2", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0203", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8717801535481093, + "fixed25_actual_fraction": 0.25299562316136903, + "fixed25_rescue": 1180, + "fixed25_harm": 180, + "rank25_metric": 0.8714213962832748, + "validation_selection": 0.8382268690919508, + "best_epoch": 3, + "seconds": 12.40796971321106 + }, + { + "id": "formal_0204", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.872067159359977, + "fixed25_actual_fraction": 0.25407189495587285, + "fixed25_rescue": 1178, + "fixed25_harm": 174, + "rank25_metric": 0.8703451244887709, + "validation_selection": 0.8385276425093097, + "best_epoch": 11, + "seconds": 27.56421136856079 + }, + { + "id": "formal_0104", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8709191361125063, + "fixed25_actual_fraction": 0.25400014350290595, + "fixed25_rescue": 1153, + "fixed25_harm": 165, + "rank25_metric": 0.8695558585061347, + "validation_selection": 0.8383414494414207, + "best_epoch": 7, + "seconds": 91.79227209091187 + } + ] + }, + { + "key": [ + "nlvr2", + "vilt", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0209", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8323886058692689, + "fixed25_actual_fraction": 0.24072612470402527, + "fixed25_rescue": 1189, + "fixed25_harm": 239, + "rank25_metric": 0.8343258950993757, + "validation_selection": 0.7950730449727872, + "best_epoch": 4, + "seconds": 18.022375106811523 + }, + { + "id": "formal_0210", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8337518834756403, + "fixed25_actual_fraction": 0.24704025256511444, + "fixed25_rescue": 1213, + "fixed25_harm": 244, + "rank25_metric": 0.8344693980053096, + "validation_selection": 0.7959610426811802, + "best_epoch": 8, + "seconds": 38.606887102127075 + }, + { + "id": "formal_0139", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8327473631341035, + "fixed25_actual_fraction": 0.24208940231039677, + "fixed25_rescue": 1196, + "fixed25_harm": 241, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7952162704096247, + "best_epoch": 3, + "seconds": 136.20692539215088 + } + ] + }, + { + "key": [ + "nlvr2", + "vilt", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0211", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8336083805697065, + "fixed25_actual_fraction": 0.24431369735237138, + "fixed25_rescue": 1234, + "fixed25_harm": 267, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7945717559438557, + "best_epoch": 2, + "seconds": 12.189603567123413 + }, + { + "id": "formal_0212", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8308818253569635, + "fixed25_actual_fraction": 0.2361340317141422, + "fixed25_rescue": 1178, + "fixed25_harm": 249, + "rank25_metric": 0.834110640740475, + "validation_selection": 0.7955027212832999, + "best_epoch": 3, + "seconds": 18.873005151748657 + }, + { + "id": "formal_0154", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8329626174930043, + "fixed25_actual_fraction": 0.24094137906292604, + "fixed25_rescue": 1225, + "fixed25_harm": 267, + "rank25_metric": 0.8348281552701442, + "validation_selection": 0.7948438842738471, + "best_epoch": 2, + "seconds": 78.18392181396484 + } + ] + }, + { + "key": [ + "construction", + "yolo26x", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0213", + "tag": "baseline_25pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22250151381741792, + "validation_selection": 0.245059255252362, + "best_epoch": 2, + "seconds": 2.557858467102051 + }, + { + "id": "formal_0214", + "tag": "baseline_50pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22250151381741792, + "validation_selection": 0.2411747600492345, + "best_epoch": 1, + "seconds": 3.123697280883789 + }, + { + "id": "formal_0169", + "tag": "baseline_100pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.22272342953859628, + "validation_selection": 0.23173775721145673, + "best_epoch": 4, + "seconds": 9.216264009475708 + } + ] + }, + { + "key": [ + "construction", + "yolo26x", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0215", + "tag": "baseline_25pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.2490976644605033, + "best_epoch": 7, + "seconds": 3.354280948638916 + }, + { + "id": "formal_0216", + "tag": "baseline_50pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.2490976644605033, + "best_epoch": 14, + "seconds": 4.801910877227783 + }, + { + "id": "formal_0177", + "tag": "baseline_100pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.24929743985848513, + "best_epoch": 13, + "seconds": 11.173577785491943 + } + ] + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0205", + "tag": "baseline_25pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12161748372078302, + "fixed25_actual_fraction": 0.26897470039946736, + "fixed25_rescue": 1103, + "fixed25_harm": 518, + "rank25_metric": 0.12103479129839005, + "validation_selection": 0.13942018347651794, + "best_epoch": 1, + "seconds": 3.9248127937316895 + }, + { + "id": "formal_0206", + "tag": "baseline_50pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.11138731664705415, + "fixed25_actual_fraction": 0.25798934753661784, + "fixed25_rescue": 1337, + "fixed25_harm": 464, + "rank25_metric": 0.11157434949576527, + "validation_selection": 0.10286119045745137, + "best_epoch": 3, + "seconds": 5.573187828063965 + }, + { + "id": "formal_0114", + "tag": "baseline_100pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.11409050364241882, + "fixed25_actual_fraction": 0.2333555259653795, + "fixed25_rescue": 1155, + "fixed25_harm": 428, + "rank25_metric": 0.1142018021781044, + "validation_selection": 0.09196748903785351, + "best_epoch": 1, + "seconds": 7.1226747035980225 + } + ] + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0207", + "tag": "baseline_25pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.14471609214350373, + "fixed25_actual_fraction": 0.2892809587217044, + "fixed25_rescue": 1383, + "fixed25_harm": 258, + "rank25_metric": 0.14346432207238047, + "validation_selection": 0.16233096000534752, + "best_epoch": 17, + "seconds": 8.028136253356934 + }, + { + "id": "formal_0208", + "tag": "baseline_50pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1499641421353766, + "fixed25_actual_fraction": 0.26797603195739017, + "fixed25_rescue": 1348, + "fixed25_harm": 225, + "rank25_metric": 0.14991013725904712, + "validation_selection": 0.14960536227871302, + "best_epoch": 14, + "seconds": 7.761091709136963 + }, + { + "id": "formal_0129", + "tag": "baseline_100pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1498069310931111, + "fixed25_actual_fraction": 0.2539946737683089, + "fixed25_rescue": 1284, + "fixed25_harm": 203, + "rank25_metric": 0.15030144907600507, + "validation_selection": 0.14151521056562297, + "best_epoch": 16, + "seconds": 13.48578929901123 + } + ] + } +] \ No newline at end of file diff --git a/paper_results/data/shared_search.json b/paper_results/data/shared_search.json new file mode 100644 index 0000000000000000000000000000000000000000..9392aa0d85468e9f65ad61fa23018b5535b43532 --- /dev/null +++ b/paper_results/data/shared_search.json @@ -0,0 +1,720 @@ +{ + "status": "search_complete", + "started_at": 1789041525.1799183, + "completed_stages": [ + "hidden_percent", + "student", + "representation_dim", + "lr", + "target", + "strategy" + ], + "selected": "shared_12", + "candidates": [ + { + "id": "shared_00", + "settings": { + "student": "4M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_00_cub_MiniCPM", + "shared_00_cub_Qwen3.8", + "shared_00_glsim_MiniCPM", + "shared_00_glsim_Qwen3.8", + "shared_00_groundingdino_MiniCPM", + "shared_00_groundingdino_Qwen3.8", + "shared_00_instancevg_MiniCPM", + "shared_00_instancevg_Qwen3.8", + "shared_00_nlvr2_MiniCPM", + "shared_00_nlvr2_Qwen3.8", + "shared_00_vilt_MiniCPM", + "shared_00_vilt_Qwen3.8", + "shared_00_yolo26x_MiniCPM", + "shared_00_yolo26x_Qwen3.8", + "shared_00_rtdetr_x_fp32_MiniCPM", + "shared_00_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9498541666666667, + "grefcoco": 0.7090097687820648, + "nlvr2": 0.8164207963334288, + "construction": 0.175986962717605 + }, + "validation_mean": 0.6628179236249413, + "mean_active_parameters": 4881924.0, + "mean_fit_seconds": 37.63353890167855 + }, + { + "id": "shared_01", + "settings": { + "student": "4M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 30 + }, + "runs": [ + "shared_01_cub_MiniCPM", + "shared_01_cub_Qwen3.8", + "shared_01_glsim_MiniCPM", + "shared_01_glsim_Qwen3.8", + "shared_01_groundingdino_MiniCPM", + "shared_01_groundingdino_Qwen3.8", + "shared_01_instancevg_MiniCPM", + "shared_01_instancevg_Qwen3.8", + "shared_01_nlvr2_MiniCPM", + "shared_01_nlvr2_Qwen3.8", + "shared_01_vilt_MiniCPM", + "shared_01_vilt_Qwen3.8", + "shared_01_yolo26x_MiniCPM", + "shared_01_yolo26x_Qwen3.8", + "shared_01_rtdetr_x_fp32_MiniCPM", + "shared_01_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9346041666666667, + "grefcoco": 0.7090431513107035, + "nlvr2": 0.8157691205958177, + "construction": 0.18298715773892013 + }, + "validation_mean": 0.660600899078027, + "mean_active_parameters": 4783620.0, + "mean_fit_seconds": 39.49221354418114 + }, + { + "id": "shared_02", + "settings": { + "student": "4M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 60 + }, + "runs": [ + "shared_02_cub_MiniCPM", + "shared_02_cub_Qwen3.8", + "shared_02_glsim_MiniCPM", + "shared_02_glsim_Qwen3.8", + "shared_02_groundingdino_MiniCPM", + "shared_02_groundingdino_Qwen3.8", + "shared_02_instancevg_MiniCPM", + "shared_02_instancevg_Qwen3.8", + "shared_02_nlvr2_MiniCPM", + "shared_02_nlvr2_Qwen3.8", + "shared_02_vilt_MiniCPM", + "shared_02_vilt_Qwen3.8", + "shared_02_yolo26x_MiniCPM", + "shared_02_yolo26x_Qwen3.8", + "shared_02_rtdetr_x_fp32_MiniCPM", + "shared_02_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9472291666666667, + "grefcoco": 0.708183990442055, + "nlvr2": 0.8155900887997709, + "construction": 0.1758169352446146 + }, + "validation_mean": 0.6617050452882768, + "mean_active_parameters": 4816388.0, + "mean_fit_seconds": 34.18548409075447 + }, + { + "id": "shared_03", + "settings": { + "student": "1M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_03_cub_MiniCPM", + "shared_03_cub_Qwen3.8", + "shared_03_glsim_MiniCPM", + "shared_03_glsim_Qwen3.8", + "shared_03_groundingdino_MiniCPM", + "shared_03_groundingdino_Qwen3.8", + "shared_03_instancevg_MiniCPM", + "shared_03_instancevg_Qwen3.8", + "shared_03_nlvr2_MiniCPM", + "shared_03_nlvr2_Qwen3.8", + "shared_03_vilt_MiniCPM", + "shared_03_vilt_Qwen3.8", + "shared_03_yolo26x_MiniCPM", + "shared_03_yolo26x_Qwen3.8", + "shared_03_rtdetr_x_fp32_MiniCPM", + "shared_03_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9559375, + "grefcoco": 0.7093348091924943, + "nlvr2": 0.8166929246634202, + "construction": 0.173092415345378 + }, + "validation_mean": 0.6637644123003231, + "mean_active_parameters": 2615822.0, + "mean_fit_seconds": 57.72181743498368 + }, + { + "id": "shared_04", + "settings": { + "student": "2M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_04_cub_MiniCPM", + "shared_04_cub_Qwen3.8", + "shared_04_glsim_MiniCPM", + "shared_04_glsim_Qwen3.8", + "shared_04_groundingdino_MiniCPM", + "shared_04_groundingdino_Qwen3.8", + "shared_04_instancevg_MiniCPM", + "shared_04_instancevg_Qwen3.8", + "shared_04_nlvr2_MiniCPM", + "shared_04_nlvr2_Qwen3.8", + "shared_04_vilt_MiniCPM", + "shared_04_vilt_Qwen3.8", + "shared_04_yolo26x_MiniCPM", + "shared_04_yolo26x_Qwen3.8", + "shared_04_rtdetr_x_fp32_MiniCPM", + "shared_04_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9448958333333333, + "grefcoco": 0.7087286527514232, + "nlvr2": 0.8167144084789458, + "construction": 0.1764152469711854 + }, + "validation_mean": 0.661688535383722, + "mean_active_parameters": 3402222.0, + "mean_fit_seconds": 57.64163886100141 + }, + { + "id": "shared_05", + "settings": { + "student": "8M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_05_cub_MiniCPM", + "shared_05_cub_Qwen3.8", + "shared_05_glsim_MiniCPM", + "shared_05_glsim_Qwen3.8", + "shared_05_groundingdino_MiniCPM", + "shared_05_groundingdino_Qwen3.8", + "shared_05_instancevg_MiniCPM", + "shared_05_instancevg_Qwen3.8", + "shared_05_nlvr2_MiniCPM", + "shared_05_nlvr2_Qwen3.8", + "shared_05_vilt_MiniCPM", + "shared_05_vilt_Qwen3.8", + "shared_05_yolo26x_MiniCPM", + "shared_05_yolo26x_Qwen3.8", + "shared_05_rtdetr_x_fp32_MiniCPM", + "shared_05_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9542916666666666, + "grefcoco": 0.7095351043643263, + "nlvr2": 0.8168110856488112, + "construction": 0.17955466204144832 + }, + "validation_mean": 0.6650481296803131, + "mean_active_parameters": 7994758.0, + "mean_fit_seconds": 46.836150376082514 + }, + { + "id": "shared_06", + "settings": { + "student": "8M", + "representation_dim": 32, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_06_cub_MiniCPM", + "shared_06_cub_Qwen3.8", + "shared_06_glsim_MiniCPM", + "shared_06_glsim_Qwen3.8", + "shared_06_groundingdino_MiniCPM", + "shared_06_groundingdino_Qwen3.8", + "shared_06_instancevg_MiniCPM", + "shared_06_instancevg_Qwen3.8", + "shared_06_nlvr2_MiniCPM", + "shared_06_nlvr2_Qwen3.8", + "shared_06_vilt_MiniCPM", + "shared_06_vilt_Qwen3.8", + "shared_06_yolo26x_MiniCPM", + "shared_06_yolo26x_Qwen3.8", + "shared_06_rtdetr_x_fp32_MiniCPM", + "shared_06_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.966125, + "grefcoco": 0.7070665542202544, + "nlvr2": 0.8169149240905185, + "construction": 0.17258963439949287 + }, + "validation_mean": 0.6656740281775665, + "mean_active_parameters": 6222718.0, + "mean_fit_seconds": 63.611915712936025 + }, + { + "id": "shared_07", + "settings": { + "student": "8M", + "representation_dim": 64, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_07_cub_MiniCPM", + "shared_07_cub_Qwen3.8", + "shared_07_glsim_MiniCPM", + "shared_07_glsim_Qwen3.8", + "shared_07_groundingdino_MiniCPM", + "shared_07_groundingdino_Qwen3.8", + "shared_07_instancevg_MiniCPM", + "shared_07_instancevg_Qwen3.8", + "shared_07_nlvr2_MiniCPM", + "shared_07_nlvr2_Qwen3.8", + "shared_07_vilt_MiniCPM", + "shared_07_vilt_Qwen3.8", + "shared_07_yolo26x_MiniCPM", + "shared_07_yolo26x_Qwen3.8", + "shared_07_rtdetr_x_fp32_MiniCPM", + "shared_07_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9693333333333334, + "grefcoco": 0.7083755007379295, + "nlvr2": 0.8167716986536809, + "construction": 0.17603867039292925 + }, + "validation_mean": 0.6676298007794683, + "mean_active_parameters": 6340854.0, + "mean_fit_seconds": 62.082761554156605 + }, + { + "id": "shared_08", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_08_cub_MiniCPM", + "shared_08_cub_Qwen3.8", + "shared_08_glsim_MiniCPM", + "shared_08_glsim_Qwen3.8", + "shared_08_groundingdino_MiniCPM", + "shared_08_groundingdino_Qwen3.8", + "shared_08_instancevg_MiniCPM", + "shared_08_instancevg_Qwen3.8", + "shared_08_nlvr2_MiniCPM", + "shared_08_nlvr2_Qwen3.8", + "shared_08_vilt_MiniCPM", + "shared_08_vilt_Qwen3.8", + "shared_08_yolo26x_MiniCPM", + "shared_08_yolo26x_Qwen3.8", + "shared_08_rtdetr_x_fp32_MiniCPM", + "shared_08_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9704166666666667, + "grefcoco": 0.7082156159955021, + "nlvr2": 0.8169006015468347, + "construction": 0.17553875298671762 + }, + "validation_mean": 0.6677679092989303, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 58.37819891100662 + }, + { + "id": "shared_09", + "settings": { + "student": "8M", + "representation_dim": 256, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_09_cub_MiniCPM", + "shared_09_cub_Qwen3.8", + "shared_09_glsim_MiniCPM", + "shared_09_glsim_Qwen3.8", + "shared_09_groundingdino_MiniCPM", + "shared_09_groundingdino_Qwen3.8", + "shared_09_instancevg_MiniCPM", + "shared_09_instancevg_Qwen3.8", + "shared_09_nlvr2_MiniCPM", + "shared_09_nlvr2_Qwen3.8", + "shared_09_vilt_MiniCPM", + "shared_09_vilt_Qwen3.8", + "shared_09_yolo26x_MiniCPM", + "shared_09_yolo26x_Qwen3.8", + "shared_09_rtdetr_x_fp32_MiniCPM", + "shared_09_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9678541666666667, + "grefcoco": 0.7090659919881931, + "nlvr2": 0.8167609567459181, + "construction": 0.17265650809871438 + }, + "validation_mean": 0.6665844058748731, + "mean_active_parameters": 7049670.0, + "mean_fit_seconds": 50.47316628225235 + }, + { + "id": "shared_10", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.0001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_10_cub_MiniCPM", + "shared_10_cub_Qwen3.8", + "shared_10_glsim_MiniCPM", + "shared_10_glsim_Qwen3.8", + "shared_10_groundingdino_MiniCPM", + "shared_10_groundingdino_Qwen3.8", + "shared_10_instancevg_MiniCPM", + "shared_10_instancevg_Qwen3.8", + "shared_10_nlvr2_MiniCPM", + "shared_10_nlvr2_Qwen3.8", + "shared_10_vilt_MiniCPM", + "shared_10_vilt_Qwen3.8", + "shared_10_yolo26x_MiniCPM", + "shared_10_yolo26x_Qwen3.8", + "shared_10_rtdetr_x_fp32_MiniCPM", + "shared_10_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9694166666666667, + "grefcoco": 0.7047965422728231, + "nlvr2": 0.8161844743626469, + "construction": 0.17403212405686308 + }, + "validation_mean": 0.66610745183975, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 91.61671701988234 + }, + { + "id": "shared_11", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.0003, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_11_cub_MiniCPM", + "shared_11_cub_Qwen3.8", + "shared_11_glsim_MiniCPM", + "shared_11_glsim_Qwen3.8", + "shared_11_groundingdino_MiniCPM", + "shared_11_groundingdino_Qwen3.8", + "shared_11_instancevg_MiniCPM", + "shared_11_instancevg_Qwen3.8", + "shared_11_nlvr2_MiniCPM", + "shared_11_nlvr2_Qwen3.8", + "shared_11_vilt_MiniCPM", + "shared_11_vilt_Qwen3.8", + "shared_11_yolo26x_MiniCPM", + "shared_11_yolo26x_Qwen3.8", + "shared_11_rtdetr_x_fp32_MiniCPM", + "shared_11_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9692083333333333, + "grefcoco": 0.7071210204511913, + "nlvr2": 0.8166034087653967, + "construction": 0.17173346213113433 + }, + "validation_mean": 0.6661665561702639, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 75.3759171601705 + }, + { + "id": "shared_12", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_12_cub_MiniCPM", + "shared_12_cub_Qwen3.8", + "shared_12_glsim_MiniCPM", + "shared_12_glsim_Qwen3.8", + "shared_12_groundingdino_MiniCPM", + "shared_12_groundingdino_Qwen3.8", + "shared_12_instancevg_MiniCPM", + "shared_12_instancevg_Qwen3.8", + "shared_12_nlvr2_MiniCPM", + "shared_12_nlvr2_Qwen3.8", + "shared_12_vilt_MiniCPM", + "shared_12_vilt_Qwen3.8", + "shared_12_yolo26x_MiniCPM", + "shared_12_yolo26x_Qwen3.8", + "shared_12_rtdetr_x_fp32_MiniCPM", + "shared_12_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9698125, + "grefcoco": 0.7082964368543117, + "nlvr2": 0.8164959896877686, + "construction": 0.1768778191883302 + }, + "validation_mean": 0.6678706864326026, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 67.41994623254868 + }, + { + "id": "shared_13", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.01, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_13_cub_MiniCPM", + "shared_13_cub_Qwen3.8", + "shared_13_glsim_MiniCPM", + "shared_13_glsim_Qwen3.8", + "shared_13_groundingdino_MiniCPM", + "shared_13_groundingdino_Qwen3.8", + "shared_13_instancevg_MiniCPM", + "shared_13_instancevg_Qwen3.8", + "shared_13_nlvr2_MiniCPM", + "shared_13_nlvr2_Qwen3.8", + "shared_13_vilt_MiniCPM", + "shared_13_vilt_Qwen3.8", + "shared_13_yolo26x_MiniCPM", + "shared_13_yolo26x_Qwen3.8", + "shared_13_rtdetr_x_fp32_MiniCPM", + "shared_13_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9509375, + "grefcoco": 0.7100024597652681, + "nlvr2": 0.8169936980807792, + "construction": 0.17572600052962756 + }, + "validation_mean": 0.6634149145939187, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 58.952753572441 + }, + { + "id": "shared_14", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "gain", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_14_cub_MiniCPM", + "shared_14_cub_Qwen3.8", + "shared_14_glsim_MiniCPM", + "shared_14_glsim_Qwen3.8", + "shared_14_groundingdino_MiniCPM", + "shared_14_groundingdino_Qwen3.8", + "shared_14_instancevg_MiniCPM", + "shared_14_instancevg_Qwen3.8", + "shared_14_nlvr2_MiniCPM", + "shared_14_nlvr2_Qwen3.8", + "shared_14_vilt_MiniCPM", + "shared_14_vilt_Qwen3.8", + "shared_14_yolo26x_MiniCPM", + "shared_14_yolo26x_Qwen3.8", + "shared_14_rtdetr_x_fp32_MiniCPM", + "shared_14_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.97975, + "grefcoco": 0.6943653805608265, + "nlvr2": 0.815819249498711, + "construction": 0.17119724948093312 + }, + "validation_mean": 0.6652829698851177, + "mean_active_parameters": 6576739.0, + "mean_fit_seconds": 46.42365771972982 + }, + { + "id": "shared_15", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "error", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_15_cub_MiniCPM", + "shared_15_cub_Qwen3.8", + "shared_15_glsim_MiniCPM", + "shared_15_glsim_Qwen3.8", + "shared_15_groundingdino_MiniCPM", + "shared_15_groundingdino_Qwen3.8", + "shared_15_instancevg_MiniCPM", + "shared_15_instancevg_Qwen3.8", + "shared_15_nlvr2_MiniCPM", + "shared_15_nlvr2_Qwen3.8", + "shared_15_vilt_MiniCPM", + "shared_15_vilt_Qwen3.8", + "shared_15_yolo26x_MiniCPM", + "shared_15_yolo26x_Qwen3.8", + "shared_15_rtdetr_x_fp32_MiniCPM", + "shared_15_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9172083333333333, + "grefcoco": 0.707660411834985, + "nlvr2": 0.8152570896591235, + "construction": 0.17163397920287538 + }, + "validation_mean": 0.6529399535075793, + "mean_active_parameters": 6576739.0, + "mean_fit_seconds": 39.98559669695533 + }, + { + "id": "shared_16", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "weighted", + "hidden_percent": 90 + }, + "runs": [ + "shared_16_cub_MiniCPM", + "shared_16_cub_Qwen3.8", + "shared_16_glsim_MiniCPM", + "shared_16_glsim_Qwen3.8", + "shared_16_groundingdino_MiniCPM", + "shared_16_groundingdino_Qwen3.8", + "shared_16_instancevg_MiniCPM", + "shared_16_instancevg_Qwen3.8", + "shared_16_nlvr2_MiniCPM", + "shared_16_nlvr2_Qwen3.8", + "shared_16_vilt_MiniCPM", + "shared_16_vilt_Qwen3.8", + "shared_16_yolo26x_MiniCPM", + "shared_16_yolo26x_Qwen3.8", + "shared_16_rtdetr_x_fp32_MiniCPM", + "shared_16_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.958625, + "grefcoco": 0.7108879752617893, + "nlvr2": 0.8163420223431682, + "construction": 0.17381623998937223 + }, + "validation_mean": 0.6649178093985824, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 39.95715373130224 + }, + { + "id": "shared_17", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "balanced", + "hidden_percent": 90 + }, + "runs": [ + "shared_17_cub_MiniCPM", + "shared_17_cub_Qwen3.8", + "shared_17_glsim_MiniCPM", + "shared_17_glsim_Qwen3.8", + "shared_17_groundingdino_MiniCPM", + "shared_17_groundingdino_Qwen3.8", + "shared_17_instancevg_MiniCPM", + "shared_17_instancevg_Qwen3.8", + "shared_17_nlvr2_MiniCPM", + "shared_17_nlvr2_Qwen3.8", + "shared_17_vilt_MiniCPM", + "shared_17_vilt_Qwen3.8", + "shared_17_yolo26x_MiniCPM", + "shared_17_yolo26x_Qwen3.8", + "shared_17_rtdetr_x_fp32_MiniCPM", + "shared_17_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9554583333333334, + "grefcoco": 0.7062583456321596, + "nlvr2": 0.8156903466055572, + "construction": 0.17510649266382114 + }, + "validation_mean": 0.6631283795587178, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 35.547329766683106 + } + ], + "failures": [], + "test_loaded": false, + "finished_at": 1789043234.371715 +} \ No newline at end of file diff --git a/paper_results/data/table_audit.json b/paper_results/data/table_audit.json new file mode 100644 index 0000000000000000000000000000000000000000..f287a772e763f1f5c9b909bd4cdf271d20dcf5fa --- /dev/null +++ b/paper_results/data/table_audit.json @@ -0,0 +1,443 @@ +{ + "counts": { + "better_metric": 11, + "fewer_calls": 15, + "both": 10, + "above_endpoints": 5, + "better_ranked_area": 15 + }, + "rows": [ + { + "key": [ + "cub", + "cub", + "MiniCPM" + ], + "validation": { + "spec": 98.58333333333333, + "vlm": 27.750000000000004, + "conf": 98.5, + "call_conf": 1.0, + "learned": 98.58333333333333, + "call_learned": 1.0, + "change_pp": 0.0, + "metric_change_pp": 0.08333333333333526 + }, + "test": { + "spec": 81.429064549534, + "vlm": 26.561960648947185, + "conf": 79.9275112185019, + "call_conf": 11.3565757680359, + "learned": 81.37728684846392, + "call_learned": 2.8822920262340355, + "change_pp": -8.474283741801864, + "metric_change_pp": 1.4497756299620246 + } + }, + { + "key": [ + "cub", + "cub", + "Qwen3.8" + ], + "validation": { + "spec": 98.58333333333333, + "vlm": 55.08333333333333, + "conf": 98.25, + "call_conf": 1.0, + "learned": 98.58333333333333, + "call_learned": 1.0, + "change_pp": 0.0, + "metric_change_pp": 0.33333333333332993 + }, + "test": { + "spec": 81.429064549534, + "vlm": 54.81532619951675, + "conf": 81.58439765274423, + "call_conf": 11.3565757680359, + "learned": 81.34276838108389, + "call_learned": 1.4325163962720056, + "change_pp": -9.924059371763894, + "metric_change_pp": -0.24162927166033743 + } + }, + { + "key": [ + "cub", + "glsim", + "MiniCPM" + ], + "validation": { + "spec": 98.91666666666666, + "vlm": 27.750000000000004, + "conf": 98.5, + "call_conf": 1.0, + "learned": 99.0, + "call_learned": 3.0, + "change_pp": 1.9999999999999998, + "metric_change_pp": 0.5000000000000004 + }, + "test": { + "spec": 90.16223679668623, + "vlm": 26.561960648947185, + "conf": 87.69416637901277, + "call_conf": 8.698653779772178, + "learned": 89.79979288919571, + "call_learned": 4.487400759406282, + "change_pp": -4.211253020365896, + "metric_change_pp": 2.105626510182945 + } + }, + { + "key": [ + "cub", + "glsim", + "Qwen3.8" + ], + "validation": { + "spec": 98.91666666666666, + "vlm": 55.08333333333333, + "conf": 98.58333333333333, + "call_conf": 1.0, + "learned": 98.83333333333333, + "call_learned": 1.0, + "change_pp": 0.0, + "metric_change_pp": 0.24999999999999467 + }, + "test": { + "spec": 90.16223679668623, + "vlm": 54.81532619951675, + "conf": 88.78149810148429, + "call_conf": 8.698653779772178, + "learned": 90.05868139454608, + "call_learned": 1.6568864342423197, + "change_pp": -7.041767345529859, + "metric_change_pp": 1.27718329306179 + } + }, + { + "key": [ + "grefcoco", + "groundingdino", + "MiniCPM" + ], + "validation": { + "spec": 56.05453650994448, + "vlm": 64.12256658935975, + "conf": 63.72197624569541, + "call_conf": 98.99501019045611, + "learned": 68.00899571298054, + "call_learned": 33.993956005341204, + "change_pp": -65.00105418511491, + "metric_change_pp": 4.287019467285125 + }, + "test": { + "spec": 26.688029946402747, + "vlm": 50.00708958398321, + "conf": 49.68380455434875, + "call_conf": 98.73805405098828, + "learned": 45.622890848764996, + "call_learned": 43.51870232254772, + "change_pp": -55.21935172844057, + "metric_change_pp": -4.060913705583752 + } + }, + { + "key": [ + "grefcoco", + "groundingdino", + "Qwen3.8" + ], + "validation": { + "spec": 56.05453650994448, + "vlm": 80.60299388572633, + "conf": 79.94237121371846, + "call_conf": 98.99501019045611, + "learned": 81.53067678684377, + "call_learned": 41.998734977862114, + "change_pp": -56.99627521259401, + "metric_change_pp": 1.5883055731253037 + }, + "test": { + "spec": 26.688029946402747, + "vlm": 74.47182599325072, + "conf": 73.87062927147436, + "call_conf": 98.73805405098828, + "learned": 66.10044522587415, + "call_learned": 57.456824433542245, + "change_pp": -41.281229617446044, + "metric_change_pp": -7.770184045600203 + } + }, + { + "key": [ + "grefcoco", + "instancevg", + "MiniCPM" + ], + "validation": { + "spec": 75.02986857825567, + "vlm": 64.12256658935975, + "conf": 77.16635041113219, + "call_conf": 64.99402628434886, + "learned": 77.29988052568699, + "call_learned": 22.995291306486752, + "change_pp": -41.99873497786211, + "metric_change_pp": 0.1335301145547918 + }, + "test": { + "spec": 66.95119530385956, + "vlm": 50.00708958398321, + "conf": 61.877889005473165, + "call_conf": 52.09426310864078, + "learned": 66.71865694921021, + "call_learned": 30.904914499617163, + "change_pp": -21.189348609023618, + "metric_change_pp": 4.840767943737056 + } + }, + { + "key": [ + "grefcoco", + "instancevg", + "Qwen3.8" + ], + "validation": { + "spec": 75.02986857825567, + "vlm": 80.60299388572633, + "conf": 82.01560193970062, + "call_conf": 70.99585353854803, + "learned": 80.81383090870757, + "call_learned": 55.99831330381615, + "change_pp": -14.997540234731888, + "metric_change_pp": -1.2017710309930374 + }, + "test": { + "spec": 66.95119530385956, + "vlm": 74.47182599325072, + "conf": 72.43286163967898, + "call_conf": 68.38612710206165, + "learned": 75.28287440093015, + "call_learned": 53.55755324277571, + "change_pp": -14.828573859285942, + "metric_change_pp": 2.850012761251175 + } + }, + { + "key": [ + "nlvr2", + "nlvr2", + "MiniCPM" + ], + "validation": { + "spec": 79.81953594958465, + "vlm": 89.90260670295045, + "conf": 90.04583213978803, + "call_conf": 85.87797192781437, + "learned": 89.94557433400172, + "call_learned": 84.98997421942137, + "change_pp": -0.8879977083930068, + "metric_change_pp": -0.10025780578630794 + }, + "test": { + "spec": 80.00287005811867, + "vlm": 90.7368874219703, + "conf": 90.77276314845376, + "call_conf": 85.80038745784601, + "learned": 90.64361053311329, + "call_learned": 84.65953935567195, + "change_pp": -1.1408481021740635, + "metric_change_pp": -0.12915261534046651 + } + }, + { + "key": [ + "nlvr2", + "nlvr2", + "Qwen3.8" + ], + "validation": { + "spec": 79.81953594958465, + "vlm": 90.50415353766829, + "conf": 90.57576625608708, + "call_conf": 85.87797192781437, + "learned": 90.9481523918648, + "call_learned": 76.99799484388427, + "change_pp": -8.879977083930102, + "metric_change_pp": 0.3723861357777247 + }, + "test": { + "spec": 80.00287005811867, + "vlm": 90.06242376408122, + "conf": 90.24180239649853, + "call_conf": 85.80038745784601, + "learned": 90.88039032790414, + "call_learned": 76.65925234986008, + "change_pp": -9.141135107985932, + "metric_change_pp": 0.6385879314056098 + } + }, + { + "key": [ + "nlvr2", + "vilt", + "MiniCPM" + ], + "validation": { + "spec": 75.56574047550845, + "vlm": 89.90260670295045, + "conf": 89.97421942136924, + "call_conf": 95.9896877685477, + "learned": 89.8166714408479, + "call_learned": 98.99742194213692, + "change_pp": 3.007734173589227, + "metric_change_pp": -0.15754798052134422 + }, + "test": { + "spec": 76.42247255506925, + "vlm": 90.7368874219703, + "conf": 90.64361053311329, + "call_conf": 95.93169261677549, + "learned": 90.70818684078353, + "call_learned": 98.96677907727631, + "change_pp": 3.0350864605008243, + "metric_change_pp": 0.0645763076702277 + } + }, + { + "key": [ + "nlvr2", + "vilt", + "Qwen3.8" + ], + "validation": { + "spec": 75.56574047550845, + "vlm": 90.50415353766829, + "conf": 90.59008879977084, + "call_conf": 95.9896877685477, + "learned": 90.41821827556575, + "call_learned": 94.98710971068462, + "change_pp": -1.0025780578630794, + "metric_change_pp": -0.17187052420509774 + }, + "test": { + "spec": 76.42247255506925, + "vlm": 90.06242376408122, + "conf": 89.96197172992753, + "call_conf": 95.93169261677549, + "learned": 89.997847456411, + "call_learned": 94.85542082227165, + "change_pp": -1.0762717945038358, + "metric_change_pp": 0.035875726483469705 + } + }, + { + "key": [ + "construction", + "yolo26x", + "MiniCPM" + ], + "validation": { + "spec": 18.254072168965788, + "vlm": 20.666016863629594, + "conf": 22.641590025257546, + "call_conf": 93.86590584878745, + "learned": 29.007041551798586, + "call_learned": 48.93009985734665, + "change_pp": -44.935805991440795, + "metric_change_pp": 6.3654515265410385 + }, + "test": { + "spec": 18.129624718318375, + "vlm": 22.361115727328066, + "conf": 25.23047077304935, + "call_conf": 92.54327563249002, + "learned": 25.82183145054675, + "call_learned": 47.20372836218375, + "change_pp": -45.33954727030626, + "metric_change_pp": 0.591360677497399 + } + }, + { + "key": [ + "construction", + "yolo26x", + "Qwen3.8" + ], + "validation": { + "spec": 18.254072168965788, + "vlm": 25.448626222520154, + "conf": 26.42878791936722, + "call_conf": 92.86733238231099, + "learned": 33.12666068966333, + "call_learned": 49.928673323823105, + "change_pp": -42.93865905848788, + "metric_change_pp": 6.697872770296115 + }, + "test": { + "spec": 18.129624718318375, + "vlm": 27.001009067886244, + "conf": 25.974900047621052, + "call_conf": 92.21038615179761, + "learned": 29.875800068449216, + "call_learned": 48.76830892143808, + "change_pp": -43.44207723035953, + "metric_change_pp": 3.9009000208281632 + } + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "MiniCPM" + ], + "validation": { + "spec": 7.641657905799261, + "vlm": 20.666016863629594, + "conf": 20.572745422879233, + "call_conf": 97.8601997146933, + "learned": 21.744085592769803, + "call_learned": 77.88873038516405, + "change_pp": -19.971469329529246, + "metric_change_pp": 1.1713401698905694 + }, + "test": { + "spec": 7.8873978710439285, + "vlm": 22.361115727328066, + "conf": 21.609512872414097, + "call_conf": 97.90279627163781, + "learned": 22.996109134220227, + "call_learned": 78.82822902796272, + "change_pp": -19.0745672436751, + "metric_change_pp": 1.3865962618061305 + } + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "Qwen3.8" + ], + "validation": { + "spec": 7.641657905799261, + "vlm": 25.448626222520154, + "conf": 24.862274646934207, + "call_conf": 98.85877318116975, + "learned": 27.197394026479483, + "call_learned": 82.88159771754636, + "change_pp": -15.977175463623395, + "metric_change_pp": 2.335119379545275 + }, + "test": { + "spec": 7.8873978710439285, + "vlm": 27.001009067886244, + "conf": 26.818431283513146, + "call_conf": 98.60186418109188, + "learned": 26.693203615113315, + "call_learned": 83.12250332889481, + "change_pp": -15.479360852197077, + "metric_change_pp": -0.1252276683998299 + } + } + ] +} \ No newline at end of file diff --git a/paper_results/figures/build_compact.py b/paper_results/figures/build_compact.py new file mode 100644 index 0000000000000000000000000000000000000000..1c8c150e49e817e2aa9dd883535f0adbc16436fc --- /dev/null +++ b/paper_results/figures/build_compact.py @@ -0,0 +1,111 @@ +"""Plot archived results without rerunning any model.""" +from pathlib import Path +import json +import numpy as np +import matplotlib +matplotlib.use("Agg") +import matplotlib.pyplot as plt +from matplotlib.patches import Patch, Rectangle +from matplotlib.ticker import MaxNLocator +root=Path(__file__).resolve().parents[1] +groups=json.loads((root/"data/results.json").read_text(encoding="utf-8"))["groups"] +scaling=json.loads((root/"data/scaling_selected.json").read_text(encoding="utf-8")) +names={"cub":"CUB-200-2011","grefcoco":"gRefCOCO","nlvr2":"NLVR2","construction":"ConstructionSite"} +plt.rcParams.update({"font.family":"serif","font.serif":["Times New Roman"],"font.size":8, + "axes.titlesize":8.5,"axes.labelsize":8,"xtick.labelsize":7,"ytick.labelsize":7,"legend.fontsize":7.5, + "pdf.fonttype":42,"ps.fonttype":42,"axes.spines.top":False,"axes.spines.right":False,"axes.linewidth":.6,"mathtext.fontset":"stix"}) +colors=["#a7c8c0","#2c8c69","#d77960","#d5d9df"] +curve_colors={"learned":"#176f91","confidence":"#c78324","small":"#536174","large":"#8a537b"} +fig,axes=plt.subplots(2,4,figsize=(7.16,2.75),gridspec_kw={"height_ratios":[.66,1.15]}) +fig.subplots_adjust(left=.064,right=.978,bottom=.19,top=.85,wspace=.30,hspace=.67) +for col,(task,expert) in enumerate(zip(names,["cub","instancevg","nlvr2","yolo26x"])): + subset=[g for g in groups if g["key"][0]==task] + chosen=next(g for g in subset if g["key"][1:]==[expert,"Qwen3.8"]) + top,ax=axes[:,col] + xs=np.array([0,1,3,4]) + floor=np.zeros(4) + for state,color in zip(["both_correct","rescuable","harmful","both_wrong"],colors): + values=np.array([100*g["opportunity"]["test"][state]/g["opportunity"]["test"]["total"] for g in subset]) + top.bar(xs,values,bottom=floor,width=.72,color=color,linewidth=0,zorder=3) + floor+=values + assert np.allclose(floor,100) + selected_bar=next(i for i,g in enumerate(subset) if g["key"]==chosen["key"]) + top.add_patch(Rectangle((xs[selected_bar]-.36,0),.72,100,fill=False,edgecolor="#263d46",lw=.8,zorder=5,clip_on=False)) + top.set(ylim=(0,101),xlim=(-.65,4.65),yticks=[0,50,100],xticks=xs) + top.set_xticklabels(["M","Q","M","Q"]) + top.tick_params(axis="x",length=0,pad=1) + for x,label in ((.5,subset[0]["expert_display"]),(3.5,subset[2]["expert_display"])): + top.text(x,-.31,label,ha="center",va="top",transform=top.get_xaxis_transform(),fontsize=6.7) + top.set_title(names[task],fontweight="bold",pad=3) + top.grid(axis="y",color="#dce1e4",lw=.4,zorder=0) + if col==0: + top.set_ylabel("Outcome (%)",labelpad=2) + else: + top.tick_params(labelleft=False) + for method,label in (("learned","Learned"),("confidence","conf.")): + saved=chosen["methods"][method] + curve=saved["test_curve"] + calls=[100*p["actual_fraction"] for p in curve] + assert np.all(np.diff(calls)>=-1e-10) + ax.plot(calls,[100*p["metric"] for p in curve],lw=1.15,color=curve_colors[method],label=label) + p=saved["test_point"] + ax.scatter(100*p["actual_fraction"],100*p["metric"],s=16,color=curve_colors[method],edgecolor="white",linewidth=.4,zorder=5,clip_on=False) + for model,label,ls in (("small","Specialist","--"),("large","VLM",":")): + ax.axhline(100*chosen[model]["test"],ls=ls,lw=.9,color=curve_colors[model],label=label) + ax.set(xlim=(0,100),xticks=[0,50,100]) + ax.set_title(chosen["expert_display"]+" / Q",fontsize=8,pad=3) + ax.yaxis.set_major_locator(MaxNLocator(nbins=3)) + ax.grid(axis="y",color="#dce1e4",lw=.4) + if col==0: + ax.set_ylabel("Accuracy (%)",labelpad=2) + elif col==3: + ax.set_ylabel("Macro-F1 (%)",labelpad=1) +fig.legend([Patch(facecolor=c) for c in colors],["Both correct","Rescue","Harm","Both wrong"],loc="upper center",bbox_to_anchor=(.51,1.01),ncol=4,frameon=False,handlelength=1.3,columnspacing=1.7) +handles,labels=axes[1,0].get_legend_handles_labels() +fig.legend(handles,labels,loc="lower center",bbox_to_anchor=(.52,-.01),ncol=4,frameon=False,handlelength=2,columnspacing=1.8) +fig.text(.52,.083,"Actual test calls (%)",ha="center",fontsize=8) +fig.savefig(root/"figures/gains_compact.pdf") +fig.savefig(root/"figures/gains_compact.png",dpi=300) +plt.close(fig) + +fig,axes=plt.subplots(2,4,figsize=(7.16,4.3),sharex=True) +fig.subplots_adjust(left=.075,right=.985,bottom=.12,top=.90,hspace=.42,wspace=.38) +for row,endpoint in enumerate(("MiniCPM","Qwen3.8")): + for col,task in enumerate(names): + ax=axes[row,col] + selected=[r for r in scaling if r["key"][0]==task and r["key"][2]==endpoint] + for entry,color,marker in zip(selected,["#176f91","#c78324"],["o","s"]): + name=next(g["expert_display"] for g in groups if g["key"]==entry["key"]) + ax.plot([25,50,100],[100*r["rank25_metric"] for r in entry["rows"]],marker=marker,lw=1.2,ms=4,color=color,label=name) + ax.set_title(names[task]+(" / M" if row==0 else " / Q"),fontweight="bold") + ax.set_xticks([25,50,100]) + ax.yaxis.set_major_locator(MaxNLocator(nbins=4)) + ax.grid(axis="y",lw=.45,color="#dce1e4") + ax.legend(loc="best",frameon=False,fontsize=6.9,handlelength=1) + if col==0: + ax.set_ylabel("Test metric (%)") +fig.suptitle("Earlier configurations: exactly 25% test calls",fontsize=11,fontweight="bold") +fig.text(.52,.035,"Router training data (%)",ha="center",fontsize=9) +fig.savefig(root/"figures/scaling_ranked.pdf") +fig.savefig(root/"figures/scaling_ranked.png",dpi=240) +plt.close(fig) + +fig,axes=plt.subplots(1,2,figsize=(7.16,3.0),sharey=True) +fig.subplots_adjust(left=.16,right=.98,top=.84,bottom=.19,wspace=.32) +for ax,endpoint in zip(axes,("MiniCPM","Qwen3.8")): + records=json.loads((root/f"data/joint_{endpoint}_comparison_20260910.json").read_text(encoding="utf-8")) + subset=[g for g in groups if g["key"][2]==endpoint] + vals=[100*next(j["delta_multi_minus_single"] for j in records if j["expert"]==g["key"][1]) for g in subset] + ax.barh(np.arange(8),vals,color=["#2c8c69" if v>=0 else "#d77960" for v in vals],height=.64) + ax.axvline(0,color="#8c949b",lw=.65) + ax.set_yticks(np.arange(8),[g["expert_display"] for g in subset]) + ax.set_title(endpoint.replace("MiniCPM","MiniCPM-V-4.5"),fontweight="bold") + ax.grid(axis="x",color="#dce1e4",lw=.4) + ax.set_axisbelow(True) + ax.set_xlim(-1.9,4.2) +axes[0].invert_yaxis() +fig.text(.55,.045,"Joint minus single-task test metric (percentage points)",ha="center",fontsize=9) +fig.savefig(root/"figures/joint_delta.pdf") +fig.savefig(root/"figures/joint_delta.png",dpi=240) +plt.close(fig) +print("Generated compact gains, matched-call data scaling, and joint-training figures.") diff --git a/paper_results/figures/build_cost.py b/paper_results/figures/build_cost.py new file mode 100644 index 0000000000000000000000000000000000000000..27f32ba7d38df3d227e5f01823f9a8208fd2494f --- /dev/null +++ b/paper_results/figures/build_cost.py @@ -0,0 +1,84 @@ +"""Restyle measured-component cost estimates; no model execution.""" +from pathlib import Path +import csv +import json +import numpy as np +import matplotlib +matplotlib.use("Agg") +import matplotlib.pyplot as plt +from matplotlib.ticker import MaxNLocator, LogLocator, NullLocator + +root = Path(__file__).resolve().parents[1] +data = root / "data/cost_20260911" +summary = json.loads((data / "cost_summary.json").read_text(encoding="utf-8")) +with (data / "full_cost_performance_curves.csv").open(encoding="utf-8-sig", newline="") as f: + rows = [r for r in csv.DictReader(f) if r["method"] in ("both", "confidence")] +methods = [("both", "learned", "#176f91", "-"), ("confidence", "conf.", "#c78324", "--")] +pairs = list(summary["components"]) +plt.rcParams.update({"font.family":"serif", "font.serif":["Times New Roman"], "font.size":8, + "axes.labelsize":8, "xtick.labelsize":7, "ytick.labelsize":7, "legend.fontsize":7.5, + "pdf.fonttype":42, "ps.fonttype":42, "axes.spines.top":False, "axes.spines.right":False, + "axes.linewidth":.6, "mathtext.fontset":"stix"}) + +def panel(ax, pair, field, log): + comp = summary["components"][pair] + for method, label, color, ls in methods: + rr = [r for r in rows if r["pair_key"] == pair and r["method"] == method] + assert len(rr) == 101 + x = np.array([float(r[field]) for r in rr]) + # Confidence does not execute the learned head; correct the shared offset. + offset = comp["router_gflops_per_input"] if "flops" in field else comp["router_latency_ms"] + if method == "confidence": + x -= offset + y = 100 * np.array([float(r["test_metric"]) for r in rr]) + sd = 100 * np.array([float(r["test_metric_std"]) for r in rr]) + assert np.all(np.diff(x) >= -1e-8) + ax.plot(x, y, color=color, lw=1.15, ls=ls, label=label) + if method == "both": + ax.fill_between(x, y - sd, y + sd, color=color, alpha=.12, lw=0) + p = next(p for p in summary["selected_points"] if p["pair_key"] == pair and p["method"] == method) + ax.scatter(p[field] - (offset if method == "confidence" else 0), 100 * p["test_metric"], s=19, + color=color, edgecolor="white", linewidth=.45, zorder=5) + if log: + ax.set_xscale("log") + ax.xaxis.set_major_locator(LogLocator(base=10, numticks=5)) + ax.xaxis.set_minor_locator(NullLocator()) + else: + ax.xaxis.set_major_locator(MaxNLocator(nbins=3)) + ax.yaxis.set_major_locator(MaxNLocator(nbins=4)) + ax.grid(axis="y", color="#dce1e4", lw=.4) + ax.set_ylabel("Accuracy (%)" if pair.startswith("grefcoco") else "Macro-F1 (%)", labelpad=2) + ax.tick_params(pad=2) + +for field, name, xlabel, log in (("flops_gflops_per_input", "flops_performance", "Estimated GFLOPs / input", True), + ("latency_ms_per_input", "latency_performance", "Estimated time / input (ms)", False)): + fig, axes = plt.subplots(1, 2, figsize=(7.16, 2.35)) + fig.subplots_adjust(left=.075, right=.99, top=.87, bottom=.28, wspace=.30) + for ax, pair, label in zip(axes, pairs, ("(a) gRefCOCO: InstanceVG / M", "(b) ConstructionSite: YOLO26x / Q")): + panel(ax, pair, field, log) + ax.set_xlabel(xlabel) + ax.text(.5, 1.08, label, transform=ax.transAxes, ha="center", fontsize=8) + handles, labels = axes[0].get_legend_handles_labels() + fig.legend(handles, labels, loc="lower center", bbox_to_anchor=(.52, -.015), ncol=2, frameon=False, handlelength=2.3) + fig.savefig(root / f"figures/{name}.pdf") + fig.savefig(root / f"figures/{name}.png", dpi=300) + plt.close(fig) + +fig, axes = plt.subplots(2, 2, figsize=(3.44, 2.30)) +fig.subplots_adjust(left=.135, right=.99, top=.88, bottom=.23, wspace=.40, hspace=.62) +for col, pair in enumerate(pairs): + for row, field in enumerate(("flops_gflops_per_input", "latency_ms_per_input")): + ax = axes[row, col] + panel(ax, pair, field, row == 0) + ax.tick_params(labelsize=6.5) + ax.yaxis.set_major_locator(MaxNLocator(nbins=3)) + ax.set_ylabel("Acc. (%)" if col == 0 else "F1 (%)", fontsize=7) + ax.set_xlabel("Est. GFLOPs / input" if row == 0 else "Est. time / input (ms)", fontsize=7, labelpad=1) + if row == 0: + ax.text(.5, 1.17, "gRefCOCO / M" if col == 0 else "ConstructionSite / Q", transform=ax.transAxes, ha="center", fontsize=7.2) +handles, labels = axes[0, 0].get_legend_handles_labels() +fig.legend(handles, labels, loc="lower center", bbox_to_anchor=(.54, .0), ncol=2, frameon=False, handlelength=2) +fig.savefig(root / "figures/cost_compact.pdf") +fig.savefig(root / "figures/cost_compact.png", dpi=300) +plt.close(fig) +print("Created two matching standalone plots and a four-panel paper figure.") diff --git a/paper_results/figures/build_tables.py b/paper_results/figures/build_tables.py new file mode 100644 index 0000000000000000000000000000000000000000..bc95f3ee9bb531d1fd5b616e89ff36d3dda7ab8c --- /dev/null +++ b/paper_results/figures/build_tables.py @@ -0,0 +1,123 @@ +"""Rebuild tables from saved results; no model execution.""" +from pathlib import Path +import json + +root = Path(__file__).resolve().parents[1] +groups = json.loads((root/"data/results.json").read_text(encoding="utf-8"))["groups"] +diagnostics = json.loads((root/"data/budget_diagnostics.json").read_text(encoding="utf-8")) +earlier = json.loads((root/"data/earlier_results.json").read_text(encoding="utf-8"))["rows"] +task_names = {"cub":"CUB-200-2011","grefcoco":"gRefCOCO","nlvr2":"NLVR2","construction":"ConstructionSite"} +joint = [] +for endpoint in ("MiniCPM","Qwen3.8"): + joint.extend(json.loads((root/f"data/joint_{endpoint}_comparison_20260910.json").read_text(encoding="utf-8"))) +rows = {name:[] for name in ("main_test","main_full","thresholds","outcomes_full","scaling_full","joint_full")} +scaling, audit = [], [] +counts = dict(better_metric=0,fewer_calls=0,both=0,above_endpoints=0,better_ranked_area=0) +for index,g in enumerate(groups): + task,expert,endpoint = g["key"] + prefix = f"{task_names[task] if index%4==0 else ''} & {g['expert_display']} & {'M' if endpoint=='MiniCPM' else 'Q'}" + if index and index%4==0: + for table in rows.values(): + table.append(r"\midrule") + blocks, raw = [], {} + for split in ("validation","test"): + conf = g["methods"]["confidence"][f"{split}_point"] + learned = g["methods"]["learned"][f"{split}_point"] + change = 100*(learned["actual_fraction"]-conf["actual_fraction"]) + macro = "callup" if change>1e-10 else "calldown" if change < -1e-10 else "samechg" + delta = rf"\{macro}{{{abs(change):.2f}}}" + metric_change = 100*(learned["metric"]-conf["metric"]) + metric_macro = "upchg" if metric_change>1e-10 else "downchg" if metric_change < -1e-10 else "samechg" + metric_delta = rf"\{metric_macro}{{{abs(metric_change):.2f}}}" + cv,lv = f"{100*conf['metric']:.2f}",f"{100*learned['metric']:.2f}" + if conf["metric"]>=learned["metric"]: + cv = rf"\textbf{{{cv}}}" + if learned["metric"]>=conf["metric"]: + lv = rf"\textbf{{{lv}}}" + if split=="test" and learned["metric"]>max(g["small"][split],g["large"][split]): + lv += r"$^\dagger$" + blocks.append(f"{100*g['small'][split]:.2f} & {100*g['large'][split]:.2f} & {cv} & {100*conf['actual_fraction']:.2f} & {lv}{metric_delta} & {100*learned['actual_fraction']:.2f}{delta}") + raw[split] = dict(spec=100*g["small"][split],vlm=100*g["large"][split],conf=100*conf["metric"],call_conf=100*conf["actual_fraction"],learned=100*learned["metric"],call_learned=100*learned["actual_fraction"],change_pp=change,metric_change_pp=metric_change) + for method in ("confidence","learned"): + p=g["methods"][method] + assert p[f"{split}_point"]["threshold"]==p["validation_point"]["threshold"] + assert p[f"{split}_point"]["operator"]==p["validation_point"]["operator"] + rows["main_test"].append(prefix+" & "+blocks[1]+r" \\") + rows["main_full"].append(prefix+" & "+" & ".join(blocks)+r" \\") + selected=g["methods"]["learned"]["test_point"] + reference=g["methods"]["confidence"]["test_point"] + better=selected["metric"]>reference["metric"] + fewer=selected["actual_fraction"]max(g["small"]["test"],g["large"]["test"]) + d=next(d for d in diagnostics if d["key"]==g["key"]) + counts["better_ranked_area"]+=d["areas"]["learned"]["test_ranked_A25_diagnostic"]>d["areas"]["confidence"]["test_ranked_A25_diagnostic"] + cutoffs=[] + for method in ("confidence","learned"): + p=g["methods"][method]["validation_point"] + op=r"\geq" if p["operator"]=="ge" else ">" + cutoffs.extend([f"{100*p['budget']:.0f}","$"+f"{op}{p['threshold']:.6g}"+"$"]) + rows["thresholds"].append(prefix+" & "+" & ".join(cutoffs)+r" \\") + opp=g["opportunity"]["test"] + keys=("both_correct","rescuable","harmful","both_wrong") + assert sum(opp[k] for k in keys)==opp["total"] + gain=100*(selected["metric"]-g["small"]["test"]) + if task!="construction": + assert abs(gain-100*(selected["rescued"]-selected["harmed"])/g["test_rows"])<1e-9 + values=[opp[k] for k in keys]+[selected[k] for k in ("rescued","harmed","missed_rescue","avoided_harm")] + rows["outcomes_full"].append(prefix+" & "+" & ".join(f"{v:,}" for v in values)+f" & {gain:+.3f}"+r" \\") + scale=[] + for fraction,tag in ((.25,"baseline_25pct"),(.5,"baseline_50pct"),(1.,"baseline_100pct")): + matches=[r for r in earlier if (r["task"],r["expert"],r["endpoint"],r["tag"])==(task,expert,endpoint,tag)] + assert len(matches)==1,(g["key"],tag,len(matches)) + assert matches[0]["fraction"]==fraction + scale.append(matches[0]) + for field in ("student","branches","fusion","target","strategy","representation_dim","hidden_layer"): + assert all(r[field]==scale[0][field] for r in scale),(g["key"],field) + rows["scaling_full"].append(prefix+" & "+" & ".join(f"{100*r['fixed25_metric']:.2f} & {100*r['fixed25_actual_fraction']:.2f} & {100*r['rank25_metric']:.2f}" for r in scale)+r" \\") + scaling.append(dict(key=g["key"],rows=scale)) + j=next(j for j in joint if (j["expert"],j["endpoint"])==(expert,endpoint)) + rows["joint_full"].append(prefix+f" & {100*j['single_task_test_fixed25']:.3f} & {100*j['multi_task_test_fixed25']:.3f} & {100*j['delta_multi_minus_single']:+.3f}"+r" \\") + audit.append(dict(key=g["key"],**raw)) +assert counts==dict(better_metric=11,fewer_calls=15,both=10,above_endpoints=5,better_ranked_area=15),counts + +head=r"\textcolor{testink}{spec.} & \textcolor{testink}{vlm.} & \textcolor{confink}{conf.} & \textcolor{confink}{call$_{\rm conf}$} & \textcolor{learnink}{learned$\uparrow$} & \textcolor{learnink}{call$_{\rm learned}\downarrow$}" +cols=r">{\columncolor{basefill}}r>{\columncolor{basefill}}r>{\columncolor{conffill}}r>{\columncolor{conffill}}r>{\columncolor{learnfill}}r>{\columncolor{learnfill}}r" +specs=[ +("main_test","table*","t","tab:main","lll"+cols, "Dataset & Specialist & VLM & "+head, + r"Test results at validation-selected thresholds. Scores and actual call rates are percentages; parentheses give learned-minus-conf. differences in percentage points. \textcolor{upgreen}{Green} marks improvement and \textcolor{downred}{red} degradation: higher scores and fewer calls are better. Bold compares routed scores; $\dagger$ exceeds both models. M: MiniCPM-V-4.5; Q: Qwen3.8-27B-FP8. ",8,10.0), +("main_full","table","H","tab:full","lll"+cols+cols, + r"\multirow{2}{*}{Task} & \multirow{2}{*}{Specialist} & \multirow{2}{*}{VLM} & \multicolumn{6}{c}{\cellcolor{valfill}\textcolor{valink}{Validation}} & \multicolumn{6}{c}{\cellcolor{testfill}\textcolor{testink}{Test}} \\ \cmidrule(lr){4-9}\cmidrule(l){10-15} & & & "+head+" & "+head, + r"Complete four-way comparison. Scores and actual call rates are percentages. Thresholds selected on the 1--99\% validation grid transfer unchanged to test. Parentheses give learned-minus-confidence differences in percentage points before rounding. Green marks improvement and red degradation: higher scores and fewer calls are better. Bold marks the higher routed score; $\dagger$ exceeds both test endpoints. M: MiniCPM-V-4.5; Q: Qwen3.8-27B-FP8.",8,1.8), +("thresholds","table","H","tab:thresholds","lllrrrr", + r"Task & Specialist & VLM & Conf. budget & Conf. cutoff & Learned budget & Learned cutoff", + "Selected nominal validation budgets and score cutoffs. The operator is part of each stored threshold. Displayed cutoffs are rounded; the archived JSON retains full precision. Budget is a percentage, not the numerical threshold.",9,4), +("outcomes_full","table","H","tab:outcomes_full","lllrrrrrrrrr", + r"Task & Specialist & VLM & BC & R & H & BW & Sel. R & Sel. H & Miss R & Avoid H & Gain (pp)", + "Test outcome counts at the learned operating point. BC/BW: both correct/wrong; R/H: rescue/harm. Selected rescues, selected harms, missed rescues and avoided harms are counted separately. Construction counts 12,016 events, with routing over 3,004 images. Metric gains are recomputed; macro-F1 is not an event-count ratio.",8.5,3), +("scaling_full","table","H","tab:scaling_full","lllrrrrrrrrr", + r"Task & Specialist & VLM & \multicolumn{3}{c}{25\% training} & \multicolumn{3}{c}{50\% training} & \multicolumn{3}{c}{100\% training}\\ \cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(l){10-12} & & & Metric & Calls & Rank & Metric & Calls & Rank & Metric & Calls & Rank", + r"Earlier training-data comparisons. Each pair retains its exploratory reference configuration; only router-training fraction changes. Within each fraction: test metric, actual test calls, and test metric at exactly 25\% ranked calls. The first two use a validation-selected 25\% threshold. These are not final shared-recipe ablations. All entries are percentages.",8.5,3), +("joint_full","table","H","tab:joint_full","lllrrr", + r"Task & Specialist & VLM & Single & Joint & Difference (pp)", + r"Earlier single-task versus joint-training comparison. Both use 4M students, image/text inputs and specialist-error supervision. Metrics use validation-selected 25\% thresholds; actual call rates are not present in this comparison file. Differences are joint minus single in percentage points. All tasks were seen during training.",9,5)] +for name,env,place,label,spec,header,caption,size,pad in specs: + content=f"\\begin{{{env}}}[{place}]\n\\centering\n\\caption{{{caption}}}\n\\label{{{label}}}\n" + content+=f"\\fontsize{{{size}}}{{{size*1.2}}}\\selectfont\n\\setlength{{\\tabcolsep}}{{{pad}pt}}\n\\renewcommand{{\\arraystretch}}{{1.12}}\n" + content+="\\begin{tabular}{@{}"+spec+"@{}}\n\\toprule\n"+header+r" \\"+"\n\\midrule\n" + content+="\n".join(rows[name])+"\n\\bottomrule\n\\end{tabular}\n"+f"\\end{{{env}}}\n" + if name == "main_test": + content = content.replace(r"\begin{table*}[t]", r"\begin{minipage}{\textwidth}") + content = content.replace(r"\caption{", r"\captionof{table}{") + content = content.replace(r"\end{table*}", r"\end{minipage}") + content = content.replace( + r">{\columncolor{learnfill}}r@{}}", + r">{\columncolor{learnfill}[\tabcolsep][0pt]}r@{}}", + ) + (root/f"tables/{name}.tex").write_text(content,encoding="utf-8") +(root/"data/table_audit.json").write_text(json.dumps(dict(counts=counts,rows=audit),indent=2),encoding="utf-8") +(root/"data/scaling_selected.json").write_text(json.dumps(scaling,indent=2),encoding="utf-8") +print(json.dumps(counts)) +print("Rebuilt all 16-pair tables from saved results.") diff --git a/paper_results/figures/cost_includes.tex b/paper_results/figures/cost_includes.tex new file mode 100644 index 0000000000000000000000000000000000000000..449b6221a92f72c3e917d8e64a921bfb794ea2f9 --- /dev/null +++ b/paper_results/figures/cost_includes.tex @@ -0,0 +1,14 @@ +% Standalone insertions for the updated three-seed router, not the revision-17 policy. +\begin{figure*}[t] +\centering +\includegraphics[width=\textwidth]{figures/flops_performance.pdf} +\caption{Test performance versus estimated computation for InstanceVG/MiniCPM and YOLO26x/Qwen. Learned uses the probability contrast $p_R-p_H$ and validation task-performance checkpoint selection; conf. uses specialist confidence. Curves average seeds 42--44; shading is one population standard deviation and dots average validation-selected policies. All thresholds are fixed before test evaluation. Costs combine measured components with actual test call fractions; VLM decoding is estimated from mean output length. Router feature encoders and unsupported operations are excluded. These are component-based estimates, not full-system counts.} +\label{fig:new_cost_flops} +\end{figure*} + +\begin{figure*}[t] +\centering +\includegraphics[width=\textwidth]{figures/latency_performance.pdf} +\caption{Test performance versus estimated amortized time per input under the same three-seed protocol. Time combines A800 specialist measurements, the routing head on cached features, and VLM batch wall time weighted by actual test calls. The VLM probe uses batch size four. Feature-encoder computation, loading, warmup and deployment queueing are excluded. This estimate is not a measurement of online per-request response latency. Learned and conf. have the same meaning as in Fig.~\ref{fig:new_cost_flops}.} +\label{fig:new_cost_time} +\end{figure*} diff --git a/paper_results/figures/gains_compact.pdf b/paper_results/figures/gains_compact.pdf new file mode 100644 index 0000000000000000000000000000000000000000..df1ad05582f86199950485af66d921ecbfd7b77f Binary files /dev/null and b/paper_results/figures/gains_compact.pdf differ diff --git a/paper_results/figures/gains_compact.png b/paper_results/figures/gains_compact.png new file mode 100644 index 0000000000000000000000000000000000000000..57f270434d4d1febfbb961266a36e65391914255 --- /dev/null +++ b/paper_results/figures/gains_compact.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cdedff2c5768301c64a47f429a77494f05a9c57b269a07ebb93c7708dcfe4b4b +size 180872 diff --git a/paper_results/figures/joint_delta.pdf b/paper_results/figures/joint_delta.pdf new file mode 100644 index 0000000000000000000000000000000000000000..d116cf06acb51981fad7df2a1359238c84fd9cae Binary files /dev/null and b/paper_results/figures/joint_delta.pdf differ diff --git a/paper_results/figures/joint_delta.png b/paper_results/figures/joint_delta.png new file mode 100644 index 0000000000000000000000000000000000000000..67c8f67dc440aac947e80b638954105ecd9cc44c Binary files /dev/null and b/paper_results/figures/joint_delta.png differ diff --git a/paper_results/figures/scaling_ranked.pdf b/paper_results/figures/scaling_ranked.pdf new file mode 100644 index 0000000000000000000000000000000000000000..4df3952ff8c612798bce982dd585839cf2a79f0f Binary files /dev/null and b/paper_results/figures/scaling_ranked.pdf differ diff --git a/paper_results/figures/scaling_ranked.png b/paper_results/figures/scaling_ranked.png new file mode 100644 index 0000000000000000000000000000000000000000..144de2f3c4d01467339320a9fe971b71137d0487 --- /dev/null +++ b/paper_results/figures/scaling_ranked.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a64ba00cdbbcdfe43f76bb5938bda44ff2adbcae6e571b5c845316dba3a29c47 +size 138869 diff --git a/paper_results/tables/capacity_main.tex b/paper_results/tables/capacity_main.tex new file mode 100644 index 0000000000000000000000000000000000000000..853270a2c90aee1efbd44507e3608c0dd3f46e1d --- /dev/null +++ b/paper_results/tables/capacity_main.tex @@ -0,0 +1,14 @@ +\begin{table}[H] +\centering +\caption{Width sweep: 8M student, lr .001, depth 90\%.} +\label{tab:capacity} +\fontsize{8}{9.6}\selectfont +\setlength{\tabcolsep}{4pt} +\begin{tabularx}{\columnwidth}{@{}l*{5}{>{\centering\arraybackslash}X}@{}} +\toprule +\rowcolor{valfill} Width & 32 & 64 & 128 & 256 & 512\\ +\midrule +$100\mathrm{Score}_{25}$ & 66.567 & 66.763 & \textbf{66.777} & 66.658 & 66.505\\ +\bottomrule +\end{tabularx} +\end{table} diff --git a/paper_results/tables/colors.tex b/paper_results/tables/colors.tex new file mode 100644 index 0000000000000000000000000000000000000000..3a6929e672bd001f56075d5b37ce077e9c0ac70c --- /dev/null +++ b/paper_results/tables/colors.tex @@ -0,0 +1,17 @@ +\definecolor{valfill}{RGB}{231,239,249} +\definecolor{testfill}{RGB}{224,239,234} +\definecolor{basefill}{RGB}{248,249,251} +\definecolor{conffill}{RGB}{255,247,231} +\definecolor{learnfill}{RGB}{235,246,240} +\definecolor{valink}{RGB}{38,83,123} +\definecolor{testink}{RGB}{34,93,82} +\definecolor{confink}{RGB}{142,91,24} +\definecolor{learnink}{RGB}{30,99,78} +\definecolor{upgreen}{RGB}{24,115,62} +\definecolor{downred}{RGB}{169,48,46} +\definecolor{samegray}{RGB}{95,102,112} +\newcommand{\upchg}[1]{\,\textcolor{upgreen}{(\ensuremath{\uparrow#1})}} +\newcommand{\downchg}[1]{\,\textcolor{downred}{(\ensuremath{\downarrow#1})}} +\newcommand{\callup}[1]{\,\textcolor{downred}{(\ensuremath{\uparrow#1})}} +\newcommand{\calldown}[1]{\,\textcolor{upgreen}{(\ensuremath{\downarrow#1})}} +\newcommand{\samechg}[1]{\,\textcolor{samegray}{(=#1)}} diff --git a/paper_results/tables/cost_records.tex b/paper_results/tables/cost_records.tex new file mode 100644 index 0000000000000000000000000000000000000000..c10fc7b0b884380d763162304c56efbebe0eca58 --- /dev/null +++ b/paper_results/tables/cost_records.tex @@ -0,0 +1,24 @@ +\begin{table}[H] +\centering +\caption{Archived partial operation counts in GFLOPs. Specialist means use 32 inputs; the range is across those inputs. Each VLM row is one fixed CUB prefill probe, not full inference. Uncovered operations are not included.} +\label{tab:cost} +\small +\setlength{\tabcolsep}{6pt} +\begin{tabular}{@{}lrrrl@{}} +\toprule +\rowcolor{valfill} Model & Inputs & Mean / probe & Range & Coverage\\ +\midrule +ResNet & 32 & 8.175 & 8.175--8.175 & Counted specialist operations\\ +GLSim & 32 & 70.281 & 70.281--70.281 & Counted specialist operations\\ +GroundingDINO & 32 & 668.259 & 499.213--820.176 & Spatial sampling uncovered\\ +InstanceVG & -- & -- & -- & Not measured\\ +BEiT3 & 32 & 76.799 & 73.587--83.260 & Counted specialist operations\\ +ViLT & 32 & 84.896 & 57.612--120.691 & Counted specialist operations\\ +YOLO26x & 32 & 143.222 & 116.224--193.928 & Counted specialist operations\\ +RT-DETR-X & 32 & 224.644 & 224.644--224.644 & Spatial sampling uncovered\\ +\midrule +MiniCPM & 1 & 33,243.606 & -- & CUB prefill only\\ +Qwen3.8 & 1 & 249,018.237 & -- & CUB prefill only\\ +\bottomrule +\end{tabular} +\end{table} diff --git a/paper_results/tables/datasets_main.tex b/paper_results/tables/datasets_main.tex new file mode 100644 index 0000000000000000000000000000000000000000..217dd51497fc4b75bb3e0cb357771a00e6235e56 --- /dev/null +++ b/paper_results/tables/datasets_main.tex @@ -0,0 +1,17 @@ +\begin{table}[H] +\centering +\caption{Datasets, tasks and dataset split sizes.} +\label{tab:data} +\fontsize{8}{9.6}\selectfont +\setlength{\tabcolsep}{2.2pt} +\begin{tabularx}{\columnwidth}{@{}l>{\raggedright\arraybackslash}Xrrr@{}} +\toprule +\rowcolor{valfill} Dataset & Task & Train & Val. & Test\\ +\midrule +CUB-200-2011 & Fine-grained classification & 4,792 & 1,200 & 5,794\\ +gRefCOCO & Semantic grounding & 209,344 & 14,229 & 35,263\\ +NLVR2 & Visual reasoning & 85,042 & 6,982 & 13,937\\ +ConstructionSite & Event detection & 6,308 & 701 & 3,004\\ +\bottomrule +\end{tabularx} +\end{table} diff --git a/paper_results/tables/joint_full.tex b/paper_results/tables/joint_full.tex new file mode 100644 index 0000000000000000000000000000000000000000..df35aafc0c99ea7eab2c3355bd4ef0faeb8f8b63 --- /dev/null +++ b/paper_results/tables/joint_full.tex @@ -0,0 +1,33 @@ +\begin{table}[H] +\centering +\caption{Earlier single-task versus joint-training comparison. Both use 4M students, image/text inputs and specialist-error supervision. Metrics use validation-selected 25\% thresholds; actual call rates are not present in this comparison file. Differences are joint minus single in percentage points. All tasks were seen during training.} +\label{tab:joint_full} +\fontsize{9}{10.799999999999999}\selectfont +\setlength{\tabcolsep}{5pt} +\renewcommand{\arraystretch}{1.12} +\begin{tabular}{@{}lllrrr@{}} +\toprule +Task & Specialist & VLM & Single & Joint & Difference (pp) \\ +\midrule +CUB-200-2011 & ResNet & M & 71.194 & 70.797 & -0.397 \\ + & ResNet & Q & 76.200 & 77.822 & +1.622 \\ + & GLSim & M & 75.923 & 77.459 & +1.536 \\ + & GLSim & Q & 83.017 & 84.484 & +1.467 \\ +\midrule +gRefCOCO & GroundingDINO & M & 35.981 & 37.005 & +1.024 \\ + & GroundingDINO & Q & 42.240 & 45.992 & +3.752 \\ + & InstanceVG & M & 63.942 & 62.437 & -1.506 \\ + & InstanceVG & Q & 72.249 & 70.530 & -1.719 \\ +\midrule +NLVR2 & BEiT3 & M & 83.468 & 83.095 & -0.373 \\ + & BEiT3 & Q & 83.727 & 83.375 & -0.352 \\ + & ViLT & M & 80.469 & 80.261 & -0.208 \\ + & ViLT & Q & 80.455 & 80.677 & +0.222 \\ +\midrule +ConstructionSite & YOLO26x & M & 18.033 & 18.362 & +0.329 \\ + & YOLO26x & Q & 19.009 & 19.603 & +0.594 \\ + & RT-DETR-X & M & 10.367 & 11.260 & +0.893 \\ + & RT-DETR-X & Q & 10.483 & 12.038 & +1.555 \\ +\bottomrule +\end{tabular} +\end{table} diff --git a/paper_results/tables/main_full.tex b/paper_results/tables/main_full.tex new file mode 100644 index 0000000000000000000000000000000000000000..cb20333881099383a5486ecd810438c0730ecee3 --- /dev/null +++ b/paper_results/tables/main_full.tex @@ -0,0 +1,33 @@ +\begin{table}[H] +\centering +\caption{Complete four-way comparison. Scores and actual call rates are percentages. Thresholds selected on the 1--99\% validation grid transfer unchanged to test. Parentheses give learned-minus-confidence differences in percentage points before rounding. Green marks improvement and red degradation: higher scores and fewer calls are better. Bold marks the higher routed score; $\dagger$ exceeds both test endpoints. M: MiniCPM-V-4.5; Q: Qwen3.8-27B-FP8.} +\label{tab:full} +\fontsize{8}{9.6}\selectfont +\setlength{\tabcolsep}{1.8pt} +\renewcommand{\arraystretch}{1.12} +\begin{tabular}{@{}lll>{\columncolor{basefill}}r>{\columncolor{basefill}}r>{\columncolor{conffill}}r>{\columncolor{conffill}}r>{\columncolor{learnfill}}r>{\columncolor{learnfill}}r>{\columncolor{basefill}}r>{\columncolor{basefill}}r>{\columncolor{conffill}}r>{\columncolor{conffill}}r>{\columncolor{learnfill}}r>{\columncolor{learnfill}}r@{}} +\toprule +\multirow{2}{*}{Task} & \multirow{2}{*}{Specialist} & \multirow{2}{*}{VLM} & \multicolumn{6}{c}{\cellcolor{valfill}\textcolor{valink}{Validation}} & \multicolumn{6}{c}{\cellcolor{testfill}\textcolor{testink}{Test}} \\ \cmidrule(lr){4-9}\cmidrule(l){10-15} & & & \textcolor{testink}{spec.} & \textcolor{testink}{vlm.} & \textcolor{confink}{conf.} & \textcolor{confink}{call$_{\rm conf}$} & \textcolor{learnink}{learned$\uparrow$} & \textcolor{learnink}{call$_{\rm learned}\downarrow$} & \textcolor{testink}{spec.} & \textcolor{testink}{vlm.} & \textcolor{confink}{conf.} & \textcolor{confink}{call$_{\rm conf}$} & \textcolor{learnink}{learned$\uparrow$} & \textcolor{learnink}{call$_{\rm learned}\downarrow$} \\ +\midrule +CUB-200-2011 & ResNet & M & 98.58 & 27.75 & 98.50 & 1.00 & \textbf{98.58}\upchg{0.08} & 1.00\samechg{0.00} & 81.43 & 26.56 & 79.93 & 11.36 & \textbf{81.38}\upchg{1.45} & 2.88\calldown{8.47} \\ + & ResNet & Q & 98.58 & 55.08 & 98.25 & 1.00 & \textbf{98.58}\upchg{0.33} & 1.00\samechg{0.00} & 81.43 & 54.82 & \textbf{81.58} & 11.36 & 81.34\downchg{0.24} & 1.43\calldown{9.92} \\ + & GLSim & M & 98.92 & 27.75 & 98.50 & 1.00 & \textbf{99.00}\upchg{0.50} & 3.00\callup{2.00} & 90.16 & 26.56 & 87.69 & 8.70 & \textbf{89.80}\upchg{2.11} & 4.49\calldown{4.21} \\ + & GLSim & Q & 98.92 & 55.08 & 98.58 & 1.00 & \textbf{98.83}\upchg{0.25} & 1.00\samechg{0.00} & 90.16 & 54.82 & 88.78 & 8.70 & \textbf{90.06}\upchg{1.28} & 1.66\calldown{7.04} \\ +\midrule +gRefCOCO & GroundingDINO & M & 56.05 & 64.12 & 63.72 & 99.00 & \textbf{68.01}\upchg{4.29} & 33.99\calldown{65.00} & 26.69 & 50.01 & \textbf{49.68} & 98.74 & 45.62\downchg{4.06} & 43.52\calldown{55.22} \\ + & GroundingDINO & Q & 56.05 & 80.60 & 79.94 & 99.00 & \textbf{81.53}\upchg{1.59} & 42.00\calldown{57.00} & 26.69 & 74.47 & \textbf{73.87} & 98.74 & 66.10\downchg{7.77} & 57.46\calldown{41.28} \\ + & InstanceVG & M & 75.03 & 64.12 & 77.17 & 64.99 & \textbf{77.30}\upchg{0.13} & 23.00\calldown{42.00} & 66.95 & 50.01 & 61.88 & 52.09 & \textbf{66.72}\upchg{4.84} & 30.90\calldown{21.19} \\ + & InstanceVG & Q & 75.03 & 80.60 & \textbf{82.02} & 71.00 & 80.81\downchg{1.20} & 56.00\calldown{15.00} & 66.95 & 74.47 & 72.43 & 68.39 & \textbf{75.28}$^\dagger$\upchg{2.85} & 53.56\calldown{14.83} \\ +\midrule +NLVR2 & BEiT3 & M & 79.82 & 89.90 & \textbf{90.05} & 85.88 & 89.95\downchg{0.10} & 84.99\calldown{0.89} & 80.00 & 90.74 & \textbf{90.77} & 85.80 & 90.64\downchg{0.13} & 84.66\calldown{1.14} \\ + & BEiT3 & Q & 79.82 & 90.50 & 90.58 & 85.88 & \textbf{90.95}\upchg{0.37} & 77.00\calldown{8.88} & 80.00 & 90.06 & 90.24 & 85.80 & \textbf{90.88}$^\dagger$\upchg{0.64} & 76.66\calldown{9.14} \\ + & ViLT & M & 75.57 & 89.90 & \textbf{89.97} & 95.99 & 89.82\downchg{0.16} & 99.00\callup{3.01} & 76.42 & 90.74 & 90.64 & 95.93 & \textbf{90.71}\upchg{0.06} & 98.97\callup{3.04} \\ + & ViLT & Q & 75.57 & 90.50 & \textbf{90.59} & 95.99 & 90.42\downchg{0.17} & 94.99\calldown{1.00} & 76.42 & 90.06 & 89.96 & 95.93 & \textbf{90.00}\upchg{0.04} & 94.86\calldown{1.08} \\ +\midrule +ConstructionSite & YOLO26x & M & 18.25 & 20.67 & 22.64 & 93.87 & \textbf{29.01}\upchg{6.37} & 48.93\calldown{44.94} & 18.13 & 22.36 & 25.23 & 92.54 & \textbf{25.82}$^\dagger$\upchg{0.59} & 47.20\calldown{45.34} \\ + & YOLO26x & Q & 18.25 & 25.45 & 26.43 & 92.87 & \textbf{33.13}\upchg{6.70} & 49.93\calldown{42.94} & 18.13 & 27.00 & 25.97 & 92.21 & \textbf{29.88}$^\dagger$\upchg{3.90} & 48.77\calldown{43.44} \\ + & RT-DETR-X & M & 7.64 & 20.67 & 20.57 & 97.86 & \textbf{21.74}\upchg{1.17} & 77.89\calldown{19.97} & 7.89 & 22.36 & 21.61 & 97.90 & \textbf{23.00}$^\dagger$\upchg{1.39} & 78.83\calldown{19.07} \\ + & RT-DETR-X & Q & 7.64 & 25.45 & 24.86 & 98.86 & \textbf{27.20}\upchg{2.34} & 82.88\calldown{15.98} & 7.89 & 27.00 & \textbf{26.82} & 98.60 & 26.69\downchg{0.13} & 83.12\calldown{15.48} \\ +\bottomrule +\end{tabular} +\end{table} diff --git a/paper_results/tables/main_test.tex b/paper_results/tables/main_test.tex new file mode 100644 index 0000000000000000000000000000000000000000..bba1399112b04629e4bfffd4630575e167e42ef3 --- /dev/null +++ b/paper_results/tables/main_test.tex @@ -0,0 +1,33 @@ +\begin{minipage}{\textwidth} +\centering +\captionof{table}{Test results at validation-selected thresholds. Scores and actual call rates are percentages; parentheses give learned-minus-conf. differences in percentage points. \textcolor{upgreen}{Green} marks improvement and \textcolor{downred}{red} degradation: higher scores and fewer calls are better. Bold compares routed scores; $\dagger$ exceeds both models. M: MiniCPM-V-4.5; Q: Qwen3.8-27B-FP8. } +\label{tab:main} +\fontsize{8}{9.6}\selectfont +\setlength{\tabcolsep}{10.0pt} +\renewcommand{\arraystretch}{1.12} +\begin{tabular}{@{}lll>{\columncolor{basefill}}r>{\columncolor{basefill}}r>{\columncolor{conffill}}r>{\columncolor{conffill}}r>{\columncolor{learnfill}}r>{\columncolor{learnfill}[\tabcolsep][0pt]}r@{}} +\toprule +Dataset & Specialist & VLM & \textcolor{testink}{spec.} & \textcolor{testink}{vlm.} & \textcolor{confink}{conf.} & \textcolor{confink}{call$_{\rm conf}$} & \textcolor{learnink}{learned$\uparrow$} & \textcolor{learnink}{call$_{\rm learned}\downarrow$} \\ +\midrule +CUB-200-2011 & ResNet & M & 81.43 & 26.56 & 79.93 & 11.36 & \textbf{81.38}\upchg{1.45} & 2.88\calldown{8.47} \\ + & ResNet & Q & 81.43 & 54.82 & \textbf{81.58} & 11.36 & 81.34\downchg{0.24} & 1.43\calldown{9.92} \\ + & GLSim & M & 90.16 & 26.56 & 87.69 & 8.70 & \textbf{89.80}\upchg{2.11} & 4.49\calldown{4.21} \\ + & GLSim & Q & 90.16 & 54.82 & 88.78 & 8.70 & \textbf{90.06}\upchg{1.28} & 1.66\calldown{7.04} \\ +\midrule +gRefCOCO & GroundingDINO & M & 26.69 & 50.01 & \textbf{49.68} & 98.74 & 45.62\downchg{4.06} & 43.52\calldown{55.22} \\ + & GroundingDINO & Q & 26.69 & 74.47 & \textbf{73.87} & 98.74 & 66.10\downchg{7.77} & 57.46\calldown{41.28} \\ + & InstanceVG & M & 66.95 & 50.01 & 61.88 & 52.09 & \textbf{66.72}\upchg{4.84} & 30.90\calldown{21.19} \\ + & InstanceVG & Q & 66.95 & 74.47 & 72.43 & 68.39 & \textbf{75.28}$^\dagger$\upchg{2.85} & 53.56\calldown{14.83} \\ +\midrule +NLVR2 & BEiT3 & M & 80.00 & 90.74 & \textbf{90.77} & 85.80 & 90.64\downchg{0.13} & 84.66\calldown{1.14} \\ + & BEiT3 & Q & 80.00 & 90.06 & 90.24 & 85.80 & \textbf{90.88}$^\dagger$\upchg{0.64} & 76.66\calldown{9.14} \\ + & ViLT & M & 76.42 & 90.74 & 90.64 & 95.93 & \textbf{90.71}\upchg{0.06} & 98.97\callup{3.04} \\ + & ViLT & Q & 76.42 & 90.06 & 89.96 & 95.93 & \textbf{90.00}\upchg{0.04} & 94.86\calldown{1.08} \\ +\midrule +ConstructionSite & YOLO26x & M & 18.13 & 22.36 & 25.23 & 92.54 & \textbf{25.82}$^\dagger$\upchg{0.59} & 47.20\calldown{45.34} \\ + & YOLO26x & Q & 18.13 & 27.00 & 25.97 & 92.21 & \textbf{29.88}$^\dagger$\upchg{3.90} & 48.77\calldown{43.44} \\ + & RT-DETR-X & M & 7.89 & 22.36 & 21.61 & 97.90 & \textbf{23.00}$^\dagger$\upchg{1.39} & 78.83\calldown{19.07} \\ + & RT-DETR-X & Q & 7.89 & 27.00 & \textbf{26.82} & 98.60 & 26.69\downchg{0.13} & 83.12\calldown{15.48} \\ +\bottomrule +\end{tabular} +\end{minipage} diff --git a/paper_results/tables/objective_main.tex b/paper_results/tables/objective_main.tex new file mode 100644 index 0000000000000000000000000000000000000000..969806f98dd4c13c20b935fccc6a2045eb29ba0e --- /dev/null +++ b/paper_results/tables/objective_main.tex @@ -0,0 +1,15 @@ +\centering +\captionof{table}{Supervision objectives (sample means minimized).} +\label{tab:objective} +\fontsize{8}{9.6}\selectfont +\setlength{\tabcolsep}{3pt} +\renewcommand{\arraystretch}{1.3} +\begin{tabularx}{\columnwidth}{@{}Xlr@{}} +\toprule +\rowcolor{valfill} Target & Objective & $100\mathrm{Score}_{25}$\\ +\midrule +Error prediction & $\operatorname{BCE}(\hat e,e)$ & 65.294\\ +Gain regression & $(\hat g-(y_R-y_H))^2$ & 66.528\\ +\rowcolor{learnfill}Four-state & $-\sum_k y_k\log p_k$ & \textbf{66.787}\\ +\bottomrule +\end{tabularx} diff --git a/paper_results/tables/outcomes_full.tex b/paper_results/tables/outcomes_full.tex new file mode 100644 index 0000000000000000000000000000000000000000..c801c5a40cd3f1e63877c3980da1e0960ece85c5 --- /dev/null +++ b/paper_results/tables/outcomes_full.tex @@ -0,0 +1,33 @@ +\begin{table}[H] +\centering +\caption{Test outcome counts at the learned operating point. BC/BW: both correct/wrong; R/H: rescue/harm. Selected rescues, selected harms, missed rescues and avoided harms are counted separately. Construction counts 12,016 events, with routing over 3,004 images. Metric gains are recomputed; macro-F1 is not an event-count ratio.} +\label{tab:outcomes_full} +\fontsize{8.5}{10.2}\selectfont +\setlength{\tabcolsep}{3pt} +\renewcommand{\arraystretch}{1.12} +\begin{tabular}{@{}lllrrrrrrrrr@{}} +\toprule +Task & Specialist & VLM & BC & R & H & BW & Sel. R & Sel. H & Miss R & Avoid H & Gain (pp) \\ +\midrule +CUB-200-2011 & ResNet & M & 1,307 & 232 & 3,411 & 844 & 11 & 14 & 221 & 3,397 & -0.052 \\ + & ResNet & Q & 2,811 & 365 & 1,907 & 711 & 5 & 10 & 360 & 1,897 & -0.086 \\ + & GLSim & M & 1,433 & 106 & 3,791 & 464 & 9 & 30 & 97 & 3,761 & -0.362 \\ + & GLSim & Q & 3,032 & 144 & 2,192 & 426 & 0 & 6 & 144 & 2,186 & -0.104 \\ +\midrule +gRefCOCO & GroundingDINO & M & 6,911 & 10,723 & 2,500 & 15,129 & 7,140 & 463 & 3,583 & 2,037 & +18.935 \\ + & GroundingDINO & Q & 7,441 & 18,820 & 1,970 & 7,032 & 14,636 & 738 & 4,184 & 1,232 & +39.412 \\ + & InstanceVG & M & 15,007 & 2,627 & 8,602 & 9,027 & 1,570 & 1,652 & 1,057 & 6,950 & -0.233 \\ + & InstanceVG & Q & 20,838 & 5,423 & 2,771 & 6,231 & 4,361 & 1,423 & 1,062 & 1,348 & +8.332 \\ +\midrule +NLVR2 & BEiT3 & M & 10,421 & 2,225 & 729 & 562 & 2,140 & 657 & 85 & 72 & +10.641 \\ + & BEiT3 & Q & 10,305 & 2,247 & 845 & 540 & 2,060 & 544 & 187 & 301 & +10.878 \\ + & ViLT & M & 9,940 & 2,706 & 711 & 580 & 2,695 & 704 & 11 & 7 & +14.286 \\ + & ViLT & Q & 9,761 & 2,791 & 890 & 495 & 2,767 & 875 & 24 & 15 & +13.575 \\ +\midrule +ConstructionSite & YOLO26x & M & 8,176 & 1,198 & 2,109 & 533 & 1,158 & 850 & 40 & 1,259 & +7.692 \\ + & YOLO26x & Q & 9,508 & 1,282 & 777 & 449 & 1,249 & 337 & 33 & 440 & +11.746 \\ + & RT-DETR-X & M & 6,060 & 3,314 & 1,945 & 697 & 3,304 & 1,470 & 10 & 475 & +15.109 \\ + & RT-DETR-X & Q & 7,329 & 3,461 & 676 & 550 & 3,448 & 576 & 13 & 100 & +18.806 \\ +\bottomrule +\end{tabular} +\end{table} diff --git a/paper_results/tables/sampling_main.tex b/paper_results/tables/sampling_main.tex new file mode 100644 index 0000000000000000000000000000000000000000..12e3ddc32ee094676f6d4a4b12d06537bd0e9fe4 --- /dev/null +++ b/paper_results/tables/sampling_main.tex @@ -0,0 +1,14 @@ +\centering +\captionof{table}{Sampling.} +\label{tab:sampling} +\fontsize{8}{9.6}\selectfont +\setlength{\tabcolsep}{3pt} +\begin{tabularx}{\columnwidth}{@{}Xr@{}} +\toprule +\rowcolor{valfill} Sampling & $100\mathrm{Score}_{25}$\\ +\midrule +Weighted loss & 66.492\\ +Balanced & 66.313\\ +\rowcolor{learnfill}Natural & \textbf{66.787}\\ +\bottomrule +\end{tabularx} diff --git a/paper_results/tables/scaling_full.tex b/paper_results/tables/scaling_full.tex new file mode 100644 index 0000000000000000000000000000000000000000..8f4e8d3c129a921e952ac128e5aa1cec76c075c1 --- /dev/null +++ b/paper_results/tables/scaling_full.tex @@ -0,0 +1,33 @@ +\begin{table}[H] +\centering +\caption{Earlier training-data comparisons. Each pair retains its exploratory reference configuration; only router-training fraction changes. Within each fraction: test metric, actual test calls, and test metric at exactly 25\% ranked calls. The first two use a validation-selected 25\% threshold. These are not final shared-recipe ablations. All entries are percentages.} +\label{tab:scaling_full} +\fontsize{8.5}{10.2}\selectfont +\setlength{\tabcolsep}{3pt} +\renewcommand{\arraystretch}{1.12} +\begin{tabular}{@{}lllrrrrrrrrr@{}} +\toprule +Task & Specialist & VLM & \multicolumn{3}{c}{25\% training} & \multicolumn{3}{c}{50\% training} & \multicolumn{3}{c}{100\% training}\\ \cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(l){10-12} & & & Metric & Calls & Rank & Metric & Calls & Rank & Metric & Calls & Rank \\ +\midrule +CUB-200-2011 & ResNet & M & 78.49 & 25.87 & 78.84 & 79.05 & 25.49 & 79.31 & 78.72 & 25.58 & 79.13 \\ + & ResNet & Q & 68.74 & 60.39 & 80.60 & 69.43 & 59.98 & 80.96 & 68.29 & 60.49 & 80.91 \\ + & GLSim & M & 86.74 & 24.82 & 86.74 & 87.00 & 24.99 & 87.00 & 86.59 & 25.51 & 86.80 \\ + & GLSim & Q & 87.97 & 23.66 & 87.76 & 87.64 & 24.53 & 87.63 & 86.85 & 25.08 & 86.85 \\ +\midrule +gRefCOCO & GroundingDINO & M & 38.16 & 24.84 & 38.20 & 36.59 & 22.03 & 37.58 & 36.93 & 22.85 & 37.58 \\ + & GroundingDINO & Q & 45.63 & 24.86 & 45.71 & 43.46 & 22.01 & 45.37 & 43.03 & 21.27 & 45.44 \\ + & InstanceVG & M & 65.23 & 34.79 & 66.53 & 65.67 & 31.24 & 66.28 & 65.48 & 33.95 & 66.56 \\ + & InstanceVG & Q & 70.98 & 10.64 & 70.20 & 74.17 & 48.24 & 72.08 & 70.90 & 12.33 & 70.22 \\ +\midrule +NLVR2 & BEiT3 & M & 86.38 & 25.39 & 86.30 & 86.83 & 24.39 & 86.90 & 87.08 & 26.12 & 86.89 \\ + & BEiT3 & Q & 87.18 & 25.30 & 87.14 & 87.21 & 25.41 & 87.03 & 87.09 & 25.40 & 86.96 \\ + & ViLT & M & 83.24 & 24.07 & 83.43 & 83.38 & 24.70 & 83.45 & 83.27 & 24.21 & 83.47 \\ + & ViLT & Q & 83.36 & 24.43 & 83.47 & 83.09 & 23.61 & 83.41 & 83.30 & 24.09 & 83.48 \\ +\midrule +ConstructionSite & YOLO26x & M & 22.26 & 18.34 & 22.25 & 22.26 & 18.34 & 22.25 & 22.28 & 18.87 & 22.27 \\ + & YOLO26x & Q & 25.04 & 18.34 & 26.33 & 25.04 & 18.34 & 26.33 & 25.04 & 18.34 & 26.33 \\ + & RT-DETR-X & M & 12.16 & 26.90 & 12.10 & 11.14 & 25.80 & 11.16 & 11.41 & 23.34 & 11.42 \\ + & RT-DETR-X & Q & 14.47 & 28.93 & 14.35 & 15.00 & 26.80 & 14.99 & 14.98 & 25.40 & 15.03 \\ +\bottomrule +\end{tabular} +\end{table} diff --git a/paper_results/tables/supp_ablation.tex b/paper_results/tables/supp_ablation.tex new file mode 100644 index 0000000000000000000000000000000000000000..15dcf61385b963209340e478e7c74cfcd6705af6 --- /dev/null +++ b/paper_results/tables/supp_ablation.tex @@ -0,0 +1,29 @@ +\begin{table}[H] +\centering +\caption{Comparisons around earlier pair-specific configurations, all paired with Q. These are not removals from the new shared recipe. V-area is $100\mathrm{Score}_{25}$ on validation. Test threshold performance and calls use the validation 25\% threshold; Test rank fixes test calls to $\lfloor0.25n\rfloor$. Values are percentages. Each block retains its own objective and training configuration. I/T/H/C/P denote image, text, hidden state, confidence and prediction summary.} +\label{tab:ablation} +\small +\setlength{\tabcolsep}{4.5pt} +\begin{tabular}{@{}llrrrr@{}} +\toprule +Pair & Earlier setting / change & V-area & Test threshold & Test calls & Test rank 25\% \\ +\midrule +ResNet/Q & P+C reference & 92.800 & 68.295 & 60.49 & 80.911 \\ + & Remove C & 97.908 & 80.652 & 24.77 & 80.652 \\ + & Remove P & 92.108 & 69.503 & 54.73 & 78.944 \\ +\midrule +GroundingDINO/Q & I+T+H+C reference & 66.509 & 43.034 & 21.27 & 45.441 \\ + & Remove I & 66.675 & 43.431 & 21.80 & 45.419 \\ + & Remove T & 66.438 & 44.171 & 22.87 & 45.532 \\ + & Remove H & 66.213 & 44.636 & 23.80 & 45.373 \\ + & Remove C & 66.462 & 42.835 & 21.18 & 45.385 \\ +\midrule +BEiT3/Q & P+C reference & 83.834 & 87.092 & 25.40 & 86.956 \\ + & Remove C & 81.813 & 80.003 & 0.00 & 83.555 \\ + & Remove P & 83.138 & 86.568 & 23.87 & 86.719 \\ +\midrule +YOLO26x/Q & P reference, width 512 & 24.930 & 25.038 & 18.34 & 26.328 \\ + & Width 32 & 24.807 & 29.177 & 43.21 & 26.310 \\ +\bottomrule +\end{tabular} +\end{table} diff --git a/paper_results/tables/supp_budget_area.tex b/paper_results/tables/supp_budget_area.tex new file mode 100644 index 0000000000000000000000000000000000000000..b87f995e747f3753121fefdef545f7bd89029412 --- /dev/null +++ b/paper_results/tables/supp_budget_area.tex @@ -0,0 +1,34 @@ +\begin{table}[H] +\centering +\caption{Low-budget test comparison in $100\mathrm{Score}_{25}$. Transferred thresholds use validation-selected budgets; exact-budget ranking selects the same number of test inputs for each router at each budget without reading test labels. The latter is a batch-ranking diagnostic, not a new deployed operating point. All thresholds, settings and ranking scores remain fixed.} +\label{tab:budgetarea} +\small +\setlength{\tabcolsep}{4pt} +\begin{tabular}{@{}lllrrrrrr@{}} +\toprule +Task & Specialist & VLM & \multicolumn{3}{c}{Transferred thresholds} & \multicolumn{3}{c}{Exact-budget test ranking} \\ +\cmidrule(lr){4-6}\cmidrule(l){7-9} + & & & Conf. & Learned & Difference & Conf. & Learned & Difference \\ +\midrule +CUB & ResNet & M & 64.729 & 77.632 & +12.903 & 78.959 & 79.676 & +0.716 \\ + & ResNet & Q & 73.162 & 80.532 & +7.370 & 81.015 & 80.637 & -0.378 \\ + & GLSim & M & 72.534 & 87.402 & +14.869 & 85.198 & 88.091 & +2.893 \\ + & GLSim & Q & 78.605 & 89.185 & +10.580 & 86.912 & 89.356 & +2.444 \\ +\midrule +gRefCOCO & GroundingDINO & M & 27.538 & 33.085 & +5.547 & 27.866 & 33.525 & +5.659 \\ + & GroundingDINO & Q & 28.625 & 35.497 & +6.872 & 29.324 & 37.352 & +8.028 \\ + & InstanceVG & M & 66.393 & 67.592 & +1.199 & 65.900 & 67.773 & +1.873 \\ + & InstanceVG & Q & 66.407 & 72.810 & +6.402 & 66.114 & 71.368 & +5.254 \\ +\midrule +NLVR2 & BEiT3 & M & 83.390 & 84.256 & +0.866 & 83.588 & 84.142 & +0.554 \\ + & BEiT3 & Q & 83.440 & 84.363 & +0.923 & 83.613 & 84.253 & +0.640 \\ + & ViLT & M & 79.874 & 80.051 & +0.177 & 80.052 & 80.152 & +0.100 \\ + & ViLT & Q & 79.887 & 80.259 & +0.372 & 80.062 & 80.375 & +0.313 \\ +\midrule +Construction & YOLO26x & M & 17.698 & 21.733 & +4.036 & 17.166 & 21.580 & +4.414 \\ + & YOLO26x & Q & 18.055 & 21.316 & +3.261 & 17.933 & 21.300 & +3.367 \\ + & RT-DETR-X & M & 8.325 & 9.727 & +1.401 & 8.357 & 9.796 & +1.438 \\ + & RT-DETR-X & Q & 8.911 & 10.227 & +1.316 & 8.908 & 10.460 & +1.552 \\ +\bottomrule +\end{tabular} +\end{table} diff --git a/paper_results/tables/supp_sensitivity.tex b/paper_results/tables/supp_sensitivity.tex new file mode 100644 index 0000000000000000000000000000000000000000..a33b2ab270ddab48949587cf32b1ea47a93d7fd5 --- /dev/null +++ b/paper_results/tables/supp_sensitivity.tex @@ -0,0 +1,31 @@ +\begin{table}[H] +\centering +\caption{Validation sensitivity in the common-recipe search. The selection score is the task-balanced mean of validation $100\mathrm{Score}_{25}$, not a pooled accuracy. Joint: four-state cross-entropy; Gain: gain regression; Error: specialist-error prediction. These eighteen settings all contributed to selection, so the differences are comparisons used to choose the final settings. Bold marks the selected recipe.} +\label{tab:sensitivity} +\small +\setlength{\tabcolsep}{5pt} +\begin{tabular}{@{}rrrrrllr@{}} +\toprule +Setting & Student & Width & Learning rate & Depth & Target & Sampling & Selection score \\ +\midrule +1 & 4M & 512 & 0.001 & 90\% & Joint & Natural & 66.2818 \\ +2 & 4M & 512 & 0.001 & 30\% & Joint & Natural & 66.0601 \\ +3 & 4M & 512 & 0.001 & 60\% & Joint & Natural & 66.1705 \\ +4 & 1M & 512 & 0.001 & 90\% & Joint & Natural & 66.3764 \\ +5 & 2M & 512 & 0.001 & 90\% & Joint & Natural & 66.1689 \\ +6 & 8M & 512 & 0.001 & 90\% & Joint & Natural & 66.5048 \\ +7 & 8M & 32 & 0.001 & 90\% & Joint & Natural & 66.5674 \\ +8 & 8M & 64 & 0.001 & 90\% & Joint & Natural & 66.7630 \\ +9 & 8M & 128 & 0.001 & 90\% & Joint & Natural & 66.7768 \\ +10 & 8M & 256 & 0.001 & 90\% & Joint & Natural & 66.6584 \\ +11 & 8M & 128 & 0.0001 & 90\% & Joint & Natural & 66.6107 \\ +12 & 8M & 128 & 0.0003 & 90\% & Joint & Natural & 66.6167 \\ +\textbf{13} & \textbf{8M} & \textbf{128} & \textbf{0.003} & \textbf{90\%} & \textbf{Joint} & \textbf{Natural} & \textbf{66.7871} \\ +14 & 8M & 128 & 0.01 & 90\% & Joint & Natural & 66.3415 \\ +15 & 8M & 128 & 0.003 & 90\% & Gain & Natural & 66.5283 \\ +16 & 8M & 128 & 0.003 & 90\% & Error & Natural & 65.2940 \\ +17 & 8M & 128 & 0.003 & 90\% & Joint & Weighted & 66.4918 \\ +18 & 8M & 128 & 0.003 & 90\% & Joint & Balanced & 66.3128 \\ +\bottomrule +\end{tabular} +\end{table} diff --git a/paper_results/tables/supp_training.tex b/paper_results/tables/supp_training.tex new file mode 100644 index 0000000000000000000000000000000000000000..fe376fe15c11082045498b088412f212c1469d50 --- /dev/null +++ b/paper_results/tables/supp_training.tex @@ -0,0 +1,20 @@ +\begin{table}[H] +\centering +\caption{Specialist provenance and training performed in this project. Student distillation and router fitting are described in the text. Reused checkpoints are not presented as models trained by this study.} +\label{tab:training} +\small +\begin{tabular}{@{}>{\raggedright\arraybackslash}p{2.3cm}>{\raggedright\arraybackslash}p{3.2cm}>{\raggedright\arraybackslash}p{11.1cm}@{}} +\toprule +Specialist & Initialization / status & Recorded training or inference settings \\ +\midrule +ResNet-50 & PyTorch pretrained weights; 200-class head trained here & Original 5,994 CUB training images; 5 epochs, batch 64, AdamW $3\!\times\!10^{-4}$, weight decay 0.05, cosine decay, label smoothing 0.1. Random $224^2$ crops and flips for training; resize 256 and center crop 224 for evaluation. The internal router-validation images were included. \\ +GLSim & Author's CUB checkpoint & Global/local model with ViT-B/16; resize 300 then center crop 224; no test-time flip. Original training provenance is not independently confirmed. \\ +GroundingDINO & Pretrained tiny checkpoint & Reused O365/GoldG/V3Det checkpoint; no project training on gRefCOCO. Fixed box threshold 0.7 and text threshold 0.25. \\ +InstanceVG & Author's gRefCOCO checkpoint & Reused task checkpoint; $320^2$ images, text limit 50 tokens, inference batch 16 and score threshold 0.7. \\ +BEiT3 & Existing NLVR2 checkpoint & Reused base patch16/224 task checkpoint; no new training. Original third-party fine-tuning logs are not available in the archive. \\ +ViLT & Existing NLVR2 checkpoint & Reused B/32 task checkpoint; ordered two-image input and text limit 40 tokens. Original fine-tuning parameters are not independently audited. \\ +YOLO26x & Pretrained; fine-tuned here & Construction 6,308/701 train/validation; $640^2$ input, batch 16, maximum 100 epochs, patience 20, seed 20260831; 92 recorded epochs, mixed precision disabled. \\ +RT-DETR-X & Pretrained; fine-tuned here & Same construction split and training limits; 39 recorded epochs. Both construction runs use automatic optimizer selection; the nominal configured learning rate 0.01 is not treated as the verified effective optimizer rate. \\ +\bottomrule +\end{tabular} +\end{table} diff --git a/paper_results/tables/thresholds.tex b/paper_results/tables/thresholds.tex new file mode 100644 index 0000000000000000000000000000000000000000..afe2c6a3e7c36e3522909c3450edbed6b70e0da5 --- /dev/null +++ b/paper_results/tables/thresholds.tex @@ -0,0 +1,33 @@ +\begin{table}[H] +\centering +\caption{Selected nominal validation budgets and score cutoffs. The operator is part of each stored threshold. Displayed cutoffs are rounded; the archived JSON retains full precision. Budget is a percentage, not the numerical threshold.} +\label{tab:thresholds} +\fontsize{9}{10.799999999999999}\selectfont +\setlength{\tabcolsep}{4pt} +\renewcommand{\arraystretch}{1.12} +\begin{tabular}{@{}lllrrrr@{}} +\toprule +Task & Specialist & VLM & Conf. budget & Conf. cutoff & Learned budget & Learned cutoff \\ +\midrule +CUB-200-2011 & ResNet & M & 1 & $\geq-0.305811$ & 1 & $\geq-0.590226$ \\ + & ResNet & Q & 1 & $\geq-0.305811$ & 1 & $\geq0.600258$ \\ + & GLSim & M & 1 & $\geq-0.48844$ & 3 & $\geq-1.56001$ \\ + & GLSim & Q & 1 & $\geq-0.48844$ & 1 & $\geq4.02929$ \\ +\midrule +gRefCOCO & GroundingDINO & M & 99 & $\geq-0.916231$ & 34 & $\geq6.07611$ \\ + & GroundingDINO & Q & 99 & $\geq-0.916231$ & 42 & $\geq5.46601$ \\ + & InstanceVG & M & 65 & $\geq-0.987771$ & 23 & $\geq-5.38178$ \\ + & InstanceVG & Q & 71 & $\geq-0.994979$ & 56 & $\geq-5.43594$ \\ +\midrule +NLVR2 & BEiT3 & M & 86 & $>-0.999739$ & 85 & $\geq-5.67533$ \\ + & BEiT3 & Q & 86 & $>-0.999739$ & 77 & $\geq-3.18357$ \\ + & ViLT & M & 96 & $\geq-0.99984$ & 99 & $\geq-4.51107$ \\ + & ViLT & Q & 96 & $\geq-0.99984$ & 95 & $\geq-7.2186$ \\ +\midrule +ConstructionSite & YOLO26x & M & 94 & $\geq-0.490234$ & 49 & $\geq-2.5235$ \\ + & YOLO26x & Q & 93 & $\geq-0.453369$ & 50 & $\geq-1.19634$ \\ + & RT-DETR-X & M & 98 & $\geq-0.422105$ & 78 & $\geq-2.93597$ \\ + & RT-DETR-X & Q & 99 & $\geq-0.441623$ & 83 & $\geq-0.421618$ \\ +\bottomrule +\end{tabular} +\end{table} diff --git a/replayed_main.csv b/replayed_main.csv new file mode 100644 index 0000000000000000000000000000000000000000..f9d080506eefdca3f4db6223e3a9a2730d3d55e0 --- /dev/null +++ b/replayed_main.csv @@ -0,0 +1,33 @@ +pair,split,metric,call_fraction +shared_12_cub_MiniCPM,val,0.9858333333333333,0.01 +shared_12_cub_MiniCPM,test,0.8137728684846393,0.028822920262340353 +shared_12_cub_Qwen3.8,val,0.9858333333333333,0.01 +shared_12_cub_Qwen3.8,test,0.8134276838108389,0.014325163962720055 +shared_12_glsim_MiniCPM,val,0.99,0.03 +shared_12_glsim_MiniCPM,test,0.8979979288919572,0.04487400759406282 +shared_12_glsim_Qwen3.8,val,0.9883333333333333,0.01 +shared_12_glsim_Qwen3.8,test,0.9005868139454608,0.016568864342423197 +shared_12_groundingdino_MiniCPM,val,0.6800899571298054,0.339939560053412 +shared_12_groundingdino_MiniCPM,test,0.45622890848764996,0.43518702322547714 +shared_12_groundingdino_Qwen3.8,val,0.8153067678684377,0.4199873497786211 +shared_12_groundingdino_Qwen3.8,test,0.6610044522587415,0.5745682443354224 +shared_12_instancevg_MiniCPM,val,0.7729988052568698,0.22995291306486754 +shared_12_instancevg_MiniCPM,test,0.6671865694921022,0.30904914499617164 +shared_12_instancevg_Qwen3.8,val,0.8081383090870757,0.5599831330381615 +shared_12_instancevg_Qwen3.8,test,0.7528287440093016,0.5355755324277571 +shared_12_nlvr2_MiniCPM,val,0.8994557433400172,0.8498997421942137 +shared_12_nlvr2_MiniCPM,test,0.906436105331133,0.8465953935567195 +shared_12_nlvr2_Qwen3.8,val,0.909481523918648,0.7699799484388428 +shared_12_nlvr2_Qwen3.8,test,0.9088039032790414,0.7665925234986009 +shared_12_vilt_MiniCPM,val,0.898166714408479,0.9899742194213692 +shared_12_vilt_MiniCPM,test,0.9070818684078352,0.9896677907727631 +shared_12_vilt_Qwen3.8,val,0.9041821827556574,0.9498710971068461 +shared_12_vilt_Qwen3.8,test,0.89997847456411,0.9485542082227165 +shared_12_yolo26x_MiniCPM,val,0.29007041551798585,0.4893009985734665 +shared_12_yolo26x_MiniCPM,test,0.2582183145054675,0.47203728362183756 +shared_12_yolo26x_Qwen3.8,val,0.33126660689663334,0.4992867332382311 +shared_12_yolo26x_Qwen3.8,test,0.29875800068449215,0.4876830892143808 +shared_12_rtdetr_x_fp32_MiniCPM,val,0.21744085592769802,0.7788873038516405 +shared_12_rtdetr_x_fp32_MiniCPM,test,0.22996109134220227,0.7882822902796272 +shared_12_rtdetr_x_fp32_Qwen3.8,val,0.2719739402647948,0.8288159771754636 +shared_12_rtdetr_x_fp32_Qwen3.8,test,0.26693203615113315,0.831225033288948 diff --git a/verification.json b/verification.json new file mode 100644 index 0000000000000000000000000000000000000000..1a344223c6d6e844e83837fea097fc39c978da32 --- /dev/null +++ b/verification.json @@ -0,0 +1,11 @@ +{ + "router_checkpoints": 16, + "specialist_checkpoints": 2, + "encoder_checkpoints": 1, + "weights_bytes": 317102080, + "nonweights_bytes_including_extracted_duplicate": 18936901, + "replayed_pairs": 16, + "validation_and_test_curves_match": true, + "github_commit": "848c5ac", + "huggingface_uploaded": false +} \ No newline at end of file diff --git a/weights.tar b/weights.tar new file mode 100644 index 0000000000000000000000000000000000000000..2d617fcaa7f7fe90223206023f1373d118b3a3c1 --- /dev/null +++ b/weights.tar @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:14fa3f2583c990fda0322980a55be2cb53adf1ca40a5ace4f172749190d0a6a2 +size 317102080 diff --git "a/\344\272\244\344\273\230\350\257\264\346\230\216.md" "b/\344\272\244\344\273\230\350\257\264\346\230\216.md" new file mode 100644 index 0000000000000000000000000000000000000000..f1934df9cc790bfdd9dbef2ba387dd6a650bc3a7 --- /dev/null +++ "b/\344\272\244\344\273\230\350\257\264\346\230\216.md" @@ -0,0 +1,16 @@ +# 权重和复现数据准备情况 + +1. 做了什么:已从 A800 下载主表的 16 个最终路由权重、YOLO26x 和 RT-DETR-X 训练权重,以及最终蒸馏编码器。归一化参数和验证阈值一同保留。 +2. 为什么做:避免只有训练代码、没有论文所用权重和结果数据,导致读者无法核对主表。 +3. 完成效果:权重约 317 MB;非权重数据低于 100 MB。已逐样本重算 16 组验证和测试曲线,结果一致;论文统计和表格重建通过。没有重新训练,也没有重新运行大模型。完整特征缓存留在 A800,没有下载。Hugging Face 尚未登录,因此尚未上传。 +4. 文件位置:本目录的 weights.tar、experiment_data.tar.gz、paper_results/;复现脚本已推送到 GitHub 的 scripts/replay_paper_results.py。具体检查结果见 verification.json。 + +## 下一步 + +在本地项目根目录的 PowerShell 运行: + +```powershell +.\tmp\hf_upload_env\Scripts\hf.exe auth login +``` + +登录后告知账号或组织名,由助手上传并补齐 GitHub 下载链接。令牌只在本地登录窗口输入,不要写入文档或聊天。